#ceph IRC Log

Index

IRC Log for 2013-11-22

Timestamps are in GMT/BST.

[0:00] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[0:05] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[0:05] * ChanServ sets mode +o joao|lap
[0:07] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[0:09] * tsnider (~tsnider@nat-216-240-30-23.netapp.com) Quit (Ping timeout: 480 seconds)
[0:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:19] * kl4m (~kl4m@66.254.36.166) Quit (Quit: Leaving...)
[0:20] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[0:20] * BillK (~BillK-OFT@124-169-202-187.dyn.iinet.net.au) Quit (Read error: Connection reset by peer)
[0:21] * andreask (~andreask@zid-vpnn042.uibk.ac.at) has joined #ceph
[0:21] * ChanServ sets mode +v andreask
[0:22] * sleinen1 (~Adium@2001:620:0:25:b04d:e9c8:aaf4:7481) Quit (Quit: Leaving.)
[0:22] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[0:23] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[0:23] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[0:25] <cjh973> does ceph-fuse know how to find the other monitors if you point it to a dead one?
[0:25] <cjh973> it seems to just hang
[0:26] * yanzheng (~zhyan@134.134.137.71) has joined #ceph
[0:26] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[0:27] * linuxkidd (~linuxkidd@ip72-193-217-254.lv.lv.cox.net) Quit (Quit: Konversation terminated!)
[0:27] * linuxkidd (~linuxkidd@ip72-193-217-254.lv.lv.cox.net) has joined #ceph
[0:28] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) Quit (Remote host closed the connection)
[0:29] * BillK (~BillK-OFT@124-169-96-192.dyn.iinet.net.au) has joined #ceph
[0:29] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit (Read error: Connection reset by peer)
[0:29] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[0:29] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Remote host closed the connection)
[0:30] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[0:30] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:30] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[0:37] <Pedras> a little clarification need here
[0:37] <Pedras> the hardware recs page used to say (I believe, could be mistaken) 1GB of RAM per osd
[0:37] <Pedras> now it says : RAM
[0:37] <Pedras> ~1GB for 1TB of storage per daemon
[0:38] <Pedras> so for a 12, 4TB hard drive the recommendation would be 48GB of RAM?
[0:38] <Pedras> ~48GB :)
[0:39] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[0:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:42] <Nats_> more or less, yes
[0:42] * aarontc (~aaron@static-50-126-79-226.hlbo.or.frontiernet.net) has joined #ceph
[0:44] * sarob_ (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[0:44] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[0:45] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[0:45] * dxd828 (~dxd828@host-92-24-127-29.ppp.as43234.net) Quit (Quit: Computer has gone to sleep.)
[0:46] * mwarwick (~mwarwick@2407:7800:200:1011:3e97:eff:fe91:d9bf) has joined #ceph
[0:47] * bcat (~bcat@64-79-127-98.static.wiline.com) Quit ()
[0:47] <Pedras> tkx Nats
[0:50] <Nats_> Pedras, it also gives you more space for cache which is good
[0:50] * rongze (~rongze@117.79.232.221) has joined #ceph
[0:51] <aarontc> hey guys, I had a kernel panic on an OSD again, this time I think the full backtrace is visible :)
[0:51] <aarontc> http://i.imgur.com/EVxjMhW.png
[0:52] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[0:53] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[0:55] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:59] <dmick> 1) ew gentoo :-P 2) that looks like "the kernel thread wedged and was evicted by the NMI watchdog"; does that seem like what happened to you?
[0:59] * rongze (~rongze@117.79.232.221) Quit (Ping timeout: 480 seconds)
[0:59] <alphe> 1 gento is fun !
[1:00] <alphe> 2 kernel i too picky :P
[1:00] <alphe> hello folks
[1:00] <alphe> I don t know if someone can help me but here is my question :
[1:00] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Quit: ...)
[1:00] <dmick> my initial suspicion would be "kernel bug".
[1:00] <alphe> I have a weird probleme with cephfs I start iton mon.0 and after a moment it jumps to another mon.2 ou mon.1 and that produce most of the data folder tree to disapear !
[1:01] <alphe> dmick or a bad compilation ... some unlinked stuff
[1:01] <aarontc> dmick: you really think it's a kernel bug? I'm happy to try another kernel
[1:01] <dmick> I don't know, but at least the arrow points in that direction for me
[1:02] * mschiff (~mschiff@85.182.236.82) Quit (Remote host closed the connection)
[1:02] <alphe> dmick how often do you here "I am happy to try another kernel" ?! gentoo users are good people !
[1:02] <aarontc> it looks more like the osd process was waiting on network receive with select, poll, or another system call that blocks, and something went wrong waking it up
[1:02] <dmick> the OSD is calling sendmsg(), and eventually the kernel unwedges it; user processes are not supposed to be able to do that
[1:04] <dmick> by "that" I mean "wedge the kernel thread just because it's in a system call"
[1:04] <alphe> aarontc never wake up a sleeping kernel taking a nap !
[1:05] <aarontc> I'm trying to figure out what you mean by wedge/unwedge, figured maybe there is another name for the concept but Google isn't helping here :)
[1:05] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[1:05] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[1:05] * mtanski (~mtanski@69.193.178.202) Quit (Ping timeout: 480 seconds)
[1:05] <alphe> nwat
[1:05] <alphe> nwat hello
[1:06] <dmick> it looks like the kernel found an internal thread sitting in a spinlock for too long, in its NMI watchdog handling
[1:06] <nwat> alphe: hello
[1:06] <dmick> and decided it was going to forcibly stop the thread
[1:06] <dmick> 'wedge' is an informal term. The kernel thread is stuck. It should not be stuck based on anything the user process does.
[1:06] <alphe> and the thread was the osd ?
[1:06] <dmick> 'stuck' is also an informal term.
[1:06] <aarontc> gotcha
[1:06] <alphe> dmick the image is accurate ...
[1:07] <aarontc> well, I use the same kernel binary on most of my machines, and only two of them are having problems
[1:07] <dmick> I'm not certain that's what the kernel is saying, but it looks like that
[1:07] * yanzheng (~zhyan@134.134.137.71) Quit (Remote host closed the connection)
[1:07] <dmick> kernel bugs like this are very very dependent on the exact workload
[1:07] <dmick> ceph uses the network harder than lots of things do
[1:07] <dmick> we've uncovered several kernel network stack bugs
[1:08] <dmick> again, it's not certain that's what's happening, but it's a good bet. Is there anything in syslog correlating with this failure?
[1:09] <aarontc> nope, the system crashed hard and didn't even shove anything out netconsole
[1:09] <dmick> yet more indication. are you running any Ceph kernel modules on those systems, or just the OSD?
[1:10] <aarontc> just OSD
[1:10] <aarontc> x6
[1:10] <dmick> the OSD simply should not be able to hang a machine.
[1:10] <aarontc> okay, well I'm gonna go powercycle it
[1:10] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[1:11] <alphe> why when I put my cephfs mounted patition on heavy duty like a recursive chown it cracks up
[1:11] <alphe> and 80% of the data folder tree disapear ?
[1:12] <alphe> that s a new behavior that i didnt have with 0.67 and 10,04 (wasn t using cephfs neither)
[1:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[1:16] <alphe> is there a reason that could make ceph-watch stuck ?
[1:16] <alphe> in logs I dont have anything specific ...
[1:16] * jluis (~JL@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[1:16] * ChanServ sets mode +o jluis
[1:17] * jluis is now known as joao|lap
[1:17] * ircolle (~Adium@2601:1:8380:2d9:e1b2:226e:c5e:41fd) Quit (Quit: Leaving.)
[1:21] <aarontc> alphe: I've been experiencing a similar issue, when I do lots of writes to cephfs, the ubuntu clients stop seeing some parts of the filesystem hierarchy
[1:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:21] <aarontc> I think it's kernel related, because none of my Gentoo machines have that problem
[1:22] <alphe> hum can be that I use the packages ceph for the raring on my saucy
[1:22] <alphe> which is not a good idea at all ...
[1:22] <aarontc> dmick: does it tell you anything that my monitoring system alerted OSD 8 journal was full, and the system was low on swap space before it went out to lunch?
[1:24] <dmick> system stress increases the likelihood of uncommon races
[1:25] <aarontc> over the course of an hour before the crash, swap space declined from 1.4GB free to < 100MB at the last data point
[1:25] <aarontc> is it normal for an OSD to report journal full?
[1:29] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[1:30] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[1:30] <aarontc> hmm, the monitoring system reported impossibly high network traffic on eth0 just before it went out to lunch
[1:30] <aarontc> 38.42Gbps
[1:32] * aarontc wishes the network was that fast
[1:34] * bcat (~bcat@64-79-127-98.static.wiline.com) has joined #ceph
[1:34] <darkfaded> hehe another monitoring coder that can't handle time offsets? :)
[1:34] <bcat> quick question, in ceph -w mode, what's op/s, does it equal to iops?
[1:35] <aarontc> darkfaded: No, the kernel paniced on that system, so whatever method Zabbix uses to collect network stats is either buggy, or a symptom of the problem
[1:35] <alphe> how to retreive ceph.keyring file with ceph-deploy ?
[1:36] <bcat> ceph-deploy gatherkeys mon01?
[1:37] <darkfaded> hmm, if the kernel actually reported the number as last thing before the crash, then it's not zabbix fault
[1:37] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[1:37] <Pedras> bcat: from an hour ago??? there were stale entries in /dev for the journal devices, so ceph-disk would be unhappy
[1:37] <darkfaded> but i'd bet on zabbix ;)
[1:38] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[1:38] <aarontc> darkfaded: well, look at the backtrace and see for yourself.. could be a bug in the network stack, as dmick thinks: http://i.imgur.com/EVxjMhW.png
[1:38] <bcat> if it's a test machine, did you zap the disks before preparing disks?
[1:38] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[1:39] <Pedras> ceph-deploy's zap disk was not happy
[1:39] <dmick> if the kernel is in that bad a shape, all bets are off
[1:39] <Pedras> so I was running the same sgdisk cmd on every disk to cleanup
[1:39] <darkfaded> dmick: :)
[1:39] <Pedras> somewhere in all that several /dev entries became stale
[1:39] <bcat> hm, i never had this issue with zap
[1:39] <bcat> here is what I did
[1:40] <aarontc> dmick: It's a vanilla kernel, with genpatches (very minor changes, basically enforces DEVTMPFS mounted at boot time), and kernel.org defconfig with addition of ext4 builtin instead of unselected
[1:40] <bcat> I created journal partitions with fdisk
[1:40] <alphe> why my cephfs stops seeing part of the filesystem on load ? how can i fix that using fuse ?
[1:40] <bcat> then zap disks, and prepare disks
[1:40] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[1:40] <alphe> but fuse is so slow T__T
[1:40] <dmick> aarontc: I'm not commenting on its quality or options, just that it exhibits signs of having gone to lunch, which happens to the best of kernels
[1:40] <alphe> is there a know kernel that work ok ?
[1:41] <alphe> emperor 0.72.1-1 ..
[1:43] <alphe> ok did a dist upgrade on the client ...
[1:44] <alphe> and got a full pack of new stuff to test !
[1:44] <bcat> Pedras, did you setup /etc/hosts on all nodes?
[1:44] <Pedras> I did not
[1:44] <Pedras> I have the "Admin" node with ceph-deploy with one node with osds and stuff
[1:44] <bcat> I did notice some of the ceph-deploy functions rely on it
[1:45] <Pedras> it is a test node and I don't have another of the kind
[1:45] <bcat> hm
[1:45] <Pedras> this particualr test is to make sure ceph actually runs on it
[1:45] <Pedras> yes indeed they do
[1:46] <bcat> can you post the error message here?
[1:46] <alphe> so far it seems it s ok
[1:46] <bcat> I think you'll just need more time to get used to ceph-deploy :) I was there
[1:47] <alphe> arg nope after a moment crack
[1:47] <alphe> chown: cannot access ?????????1014195/2?????????: No such file or directory
[1:47] <bcat> took me couple days to realize "ceph-deploy new node1" referred to monitor, not osd node
[1:48] <bcat> I was confused by the official doc at first
[1:49] <Pedras> the error is gone after making sure partx is run after parted and the dev entries are indeed what they should be, block devs
[1:49] * yanzheng (~zhyan@jfdmzpr02-ext.jf.intel.com) has joined #ceph
[1:50] <bcat> Pedras, just keep trying, i am sure ceph-deploy works fine, more than fine
[1:51] <bcat> I did have some glitches with ceph-deploy, but it was due to my settings or other issues
[1:52] <Pedras> well, I have recipe which I have followed for the last 3 clusters
[1:52] <bcat> try to do a complete purge
[1:52] <Pedras> usually version change??? yeah purge is your friend
[1:53] <bcat> oh, does anyone know what's op/s in the watch mode
[1:53] <bcat> op/s = iops?
[1:54] <Pedras> was this 1GB RAM/ TB a recent change on the web site?
[1:55] <bcat> I think so, when I looked at it couple days ago, it was 1GB ram per OSD daemon
[1:57] <Pedras> unh
[1:57] <Pedras> quite a difference
[1:57] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[1:59] <alphe> hum I wonder if find . | xargs chown user:user will explode completly cephfs
[2:00] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:02] <bcat> I just copied 3 millions files from /usr/share/doc to a kernel rbd mount point, took it 600 seconds, to create 1 file cost 2 iops, does it mean my cluster has a 3000000/600x2 iops ~ 10000 iops capacity???
[2:03] <bcat> I still wonder whether op/s = iops
[2:03] <Pedras> indeed
[2:03] <kraken> http://i.imgur.com/bQcbpki.gif
[2:03] <bcat> wow, that's pretty cool
[2:03] <Pedras> di cephio
[2:04] * linuxkidd (~linuxkidd@ip72-193-217-254.lv.lv.cox.net) Quit (Quit: Konversation terminated!)
[2:04] <gregsfortytwo1> your filesystem is batching ops, and so is rbd ;)
[2:04] <bcat> thanks guys! good luck Pedras
[2:05] <Pedras> cheers mate
[2:05] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[2:07] <bcat> gregsfortytwo1, what do you mean by "batching ops"
[2:07] <gregsfortytwo1> it's combining a bunch of creates into a small number of writes to the hard drive
[2:07] <gregsfortytwo1> making use of cache
[2:08] <gregsfortytwo1> trying to turn a bunch of small random operations into large sequential ones without totally destroying the later read performance is what modern filesystem design is all about
[2:08] <bcat> so it should not be considered as *real iops
[2:09] <gregsfortytwo1> probably not
[2:09] <bcat> sigh, I was excited for a while, though the direct IO is around 35MB/s
[2:11] <bcat> gregsfortytwo1, can you tell me the fact of this log, 2013-11-21 16:32:50.098974 mon.0 [INF] pgmap v2815: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 147 MB/s wr, 5827 op/s
[2:11] <bcat> especially the op/s
[2:11] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[2:12] <bcat> first, is 147MB/s wr a real stat?
[2:12] <JoeGruher> does ceph read-cache data? if i have 100GB of RAM on my OSD servers and I'm using a 20GB data set for a read workload am I eventually going to be reading it out of memory or will ceph fetch from disk on every read?
[2:13] <Pedras> bcat : how many disks/nodes/net bw ?
[2:13] <gregsfortytwo1> bcat: oh, yeah, that's outputting reasonable estimates
[2:13] * bandrus (~Adium@107.216.171.224) has joined #ceph
[2:14] <bcat> Pedras, I have 4 nodes, each nodes has 2 disks for OS, 2 SSDs, 6x3T 6gb/s SATA
[2:14] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) has joined #ceph
[2:14] <gregsfortytwo1> those stats aren't super-dynamic and can be a little misleading if you only look at one sample, but if they're consistent over several pgmaps then they're good
[2:14] <bcat> gregsfortytwo1, so 5827 op/s = 5827 iops?
[2:15] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[2:15] <bcat> gregsfortytwo1, that's the highest stat I've ever seen, most of them are around 2000 op/s
[2:16] <gregsfortytwo1> so the figures you're seeing there are aggregates of per-PG reports the OSDs are making on a periodic basis, so sometimes you can see some weird artifacts in terms of ops being reported in one time bucket when they were actually part of a previous bucket
[2:17] <bcat> i see
[2:17] * sjusthm (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Remote host closed the connection)
[2:17] <gregsfortytwo1> so if you see a seesaw of 0 and 10K from second to second you're probably running ~5k regularly, but just have some odd timing in the reports
[2:17] <gregsfortytwo1> if you're getting 5-6k for several seconds, you're running those numbers
[2:18] <bcat> haha, then I should happy with what you are saying
[2:18] <bcat> 2013-11-21 16:32:49.019085 mon.0 [INF] pgmap v2814: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 126 MB/s wr, 5599 op/s
[2:18] <bcat> 2013-11-21 16:32:50.098974 mon.0 [INF] pgmap v2815: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 147 MB/s wr, 5827 op/s
[2:18] <bcat> 2013-11-21 16:32:51.134344 mon.0 [INF] pgmap v2816: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 105 MB/s wr, 4199 op/s
[2:18] <bcat> 2013-11-21 16:32:52.221514 mon.0 [INF] pgmap v2817: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 40404 kB/s wr, 1518 op/s
[2:18] <bcat> 2013-11-21 16:32:53.315530 mon.0 [INF] pgmap v2818: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 109 MB/s wr, 4451 op/s
[2:18] <bcat> 2013-11-21 16:32:54.403372 mon.0 [INF] pgmap v2819: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 156 MB/s wr, 6813 op/s
[2:18] <bcat> 2013-11-21 16:32:55.447430 mon.0 [INF] pgmap v2820: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 131 MB/s wr, 5858 op/s
[2:18] <bcat> 2013-11-21 16:32:56.537898 mon.0 [INF] pgmap v2821: 192 pgs: 192 active+clean; 102 GB data, 138 GB used, 66897 GB / 67035 GB avail; 74994 kB/s wr, 3124 op/s
[2:18] <gregsfortytwo1> with 8 SSDs and 2x replication on a real workload like you ran, I think that is a believable number of ops
[2:20] <pmatulis> doesn't look like 2x replication
[2:20] <bcat> it is size 2, actual data is around 68G
[2:21] <alphe> 192 pgs: 192 not enough pgs ...
[2:21] <pmatulis> i thought you multiply the 'data' value by the replication factor to arrive at the 'used' value (plus metadata)
[2:22] <alphe> you granularity is too small that as an impact on perf normally ...
[2:22] <bcat> I did delete all other pools
[2:22] <bcat> only one pool for this testing
[2:23] <alphe> hum it is rdb so you need that pool
[2:23] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:23] <bcat> alphe, so I need data and metadata pools for the rbd??
[2:23] <gregsfortytwo1> no
[2:23] <gregsfortytwo1> you don't
[2:23] <alphe> no
[2:24] <bcat> I read the doc, it recommends around 100 pgs per osd
[2:24] <alphe> but even 192 is it small pgs ...
[2:24] <bcat> in my settings, 1 pg ~ 30GB of data
[2:24] <alphe> if you have 20 osd that make around 60 pgs per osd ...
[2:25] <pmatulis> bcat: so how many OSDs do you have?
[2:25] <bcat> 24
[2:25] <bcat> 24 OSD + 8 ssd as journal
[2:25] <pmatulis> 24 x 100 / 2 = 1200 no?
[2:25] <bcat> yes
[2:25] <alphe> hehehe ceph is making ssd price do to the floor hehehe
[2:25] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:26] <alphe> hehehe ceph is making ssd price go to the floor hehehe
[2:26] <bcat> i am confused now, so what's the recommendation pg num in my case
[2:27] * andreask (~andreask@zid-vpnn042.uibk.ac.at) Quit (Ping timeout: 480 seconds)
[2:27] <alphe> bcat according to the spacy official doc the formula is osd number * 100 / replica number
[2:27] <bcat> what if I have multiple pools?
[2:28] <alphe> no impact ...
[2:28] <bcat> so each pool need to be set to 1200 in my case?
[2:28] <alphe> it is a indexation more or less of the objects file items you have in the ceph cluster
[2:29] <alphe> more pgs means more each to find object to make them available and to remember only the active part of your cluster
[2:29] <alphe> something like that ..
[2:29] <bcat> root@ceph01:/var/lib/ceph/osd/ceph-0# ceph pg dump | cut -f1,9,13,14 | grep "]" | grep "," | tail
[2:29] <bcat> dumped all in format plain
[2:29] <bcat> 2.14 active+clean [12,3] [12,3]
[2:29] <bcat> 2.17 active+clean [16,10] [16,10]
[2:29] <bcat> 2.16 active+clean [12,11] [12,11]
[2:29] <bcat> 2.11 active+clean [15,5] [15,5]
[2:29] <bcat> 2.10 active+clean [11,12] [11,12]
[2:29] <bcat> 2.13 active+clean [3,7] [3,7]
[2:30] <bcat> 2.12 active+clean [21,16] [21,16]
[2:30] <bcat> 2.d active+clean [7,21] [7,21]
[2:30] <bcat> 2.c active+clean [11,20] [11,20]
[2:30] <bcat> 2.f active+clean [20,5] [20,5]
[2:30] <alphe> bcat flood no please ...
[2:30] <alphe> use pastebin ...(well i have the tendency to do that too and get wip cracked each time... no hard filling"
[2:31] <bcat> sure no
[2:31] <alphe> ok so for my cephfs filesystem tree that goes randomly in hollydays there is no solutions ?
[2:32] * bcat (~bcat@64-79-127-98.static.wiline.com) Quit (Remote host closed the connection)
[2:34] <alphe> ok going to bed
[2:34] <alphe> bye bye all
[2:34] * alphe (~alphe@0001ac6f.user.oftc.net) Quit (Quit: Leaving)
[2:35] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[2:35] * ChanServ sets mode +v andreask
[2:40] * sarob_ (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[2:41] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[2:43] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[2:43] * andreask (~andreask@zid-vpnn048.uibk.ac.at) has joined #ceph
[2:43] * ChanServ sets mode +v andreask
[2:45] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[2:46] * Cube (~Cube@12.248.40.138) Quit (Ping timeout: 480 seconds)
[2:48] * sarob_ (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[2:51] * alram (~alram@cpe-76-167-50-51.socal.res.rr.com) Quit (Quit: leaving)
[2:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:53] * andreask (~andreask@zid-vpnn048.uibk.ac.at) Quit (Ping timeout: 480 seconds)
[2:58] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[3:00] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[3:00] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[3:02] * yy-nm (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[3:03] * rongze (~rongze@211.155.113.217) has joined #ceph
[3:03] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[3:06] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[3:06] * yy-nm1 (~Thunderbi@122.224.154.38) has joined #ceph
[3:11] * angdraug (~angdraug@64-79-127-122.static.wiline.com) Quit (Quit: Leaving)
[3:11] * yy-nm (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[3:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[3:12] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[3:14] * yy-nm1 (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[3:17] * xarses (~andreww@64-79-127-122.static.wiline.com) Quit (Ping timeout: 480 seconds)
[3:22] * shang (~ShangWu@118.175.162.84) has joined #ceph
[3:23] * aliguori (~anthony@74.202.210.82) Quit (Remote host closed the connection)
[3:23] * yy-nm (~Thunderbi@122.224.154.38) Quit (Read error: Connection reset by peer)
[3:24] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[3:27] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) Quit (Remote host closed the connection)
[3:30] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[3:30] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:32] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[3:32] * yy-nm (~Thunderbi@122.224.154.38) Quit (Read error: Connection reset by peer)
[3:34] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[3:39] <yy-nm> hay, folks. when i run rbd ls, i get segmentation fault. and the version of ceph is 0.61.8
[3:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[3:48] <yy-nm> as same as rados. there is error message http://paste.ubuntu.com/6456572/
[3:52] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[3:55] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:56] * yy-nm (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[3:58] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[3:58] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[3:58] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[3:59] * sarob_ (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[3:59] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[4:00] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[4:04] * shang_ (~ShangWu@118.175.161.106) has joined #ceph
[4:06] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[4:06] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[4:07] * sarob_ (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[4:07] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[4:07] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[4:07] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[4:12] * shang (~ShangWu@118.175.162.84) Quit (Ping timeout: 480 seconds)
[4:15] * yy-nm (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[4:15] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[4:16] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[4:16] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[4:20] * shang_ (~ShangWu@118.175.161.106) Quit (Ping timeout: 480 seconds)
[4:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:21] * yy-nm1 (~Thunderbi@122.224.154.38) has joined #ceph
[4:22] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[4:24] * yy-nm (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[4:29] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[4:31] * yy-nm1 (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[4:33] * yy-nm1 (~Thunderbi@122.224.154.38) has joined #ceph
[4:36] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[4:37] * yy-nm (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[4:39] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[4:41] * yy-nm1 (~Thunderbi@122.224.154.38) Quit (Ping timeout: 480 seconds)
[4:42] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[4:46] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:51] * sarob_ (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[4:51] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[4:51] * mwarwick (~mwarwick@2407:7800:200:1011:3e97:eff:fe91:d9bf) Quit (Ping timeout: 480 seconds)
[4:58] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[5:04] * haomaiwa_ (~haomaiwan@118.186.151.57) has joined #ceph
[5:04] * haomaiwang (~haomaiwan@211.155.113.217) Quit (Read error: Connection reset by peer)
[5:05] * fireD_ (~fireD@93-139-168-145.adsl.net.t-com.hr) has joined #ceph
[5:06] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[5:07] * fireD (~fireD@93-142-209-202.adsl.net.t-com.hr) Quit (Ping timeout: 480 seconds)
[5:09] * shang (~ShangWu@118.175.165.81) has joined #ceph
[5:11] * Hakisho (~Hakisho@0001be3c.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:11] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[5:12] * Hakisho (~Hakisho@0001be3c.user.oftc.net) has joined #ceph
[5:18] * haomaiwa_ (~haomaiwan@118.186.151.57) Quit (Remote host closed the connection)
[5:18] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[5:19] * haomaiwang (~haomaiwan@118.186.151.57) has joined #ceph
[5:24] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[5:24] * haomaiwang (~haomaiwan@118.186.151.57) Quit (Remote host closed the connection)
[5:26] * haomaiwang (~haomaiwan@117.79.232.197) has joined #ceph
[5:26] * haomaiwang (~haomaiwan@117.79.232.197) Quit (Remote host closed the connection)
[5:27] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[5:28] * haomaiwang (~haomaiwan@199.30.140.94) has joined #ceph
[5:29] * haomaiwa_ (~haomaiwan@199.30.140.94) has joined #ceph
[5:29] * haomaiwang (~haomaiwan@199.30.140.94) Quit (Read error: Connection reset by peer)
[5:30] * haomaiwang (~haomaiwan@117.79.232.229) has joined #ceph
[5:35] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[5:37] * haomaiwa_ (~haomaiwan@199.30.140.94) Quit (Ping timeout: 480 seconds)
[5:38] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:39] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[5:39] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[5:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[5:53] * rongze (~rongze@211.155.113.217) Quit (Remote host closed the connection)
[5:59] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[6:06] * blahnana (~bman@blahnana.com) has joined #ceph
[6:07] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[6:08] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[6:13] * AfC (~andrew@2407:7800:200:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[6:17] * sarob_ (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[6:18] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[6:23] * rongze (~rongze@117.79.232.197) has joined #ceph
[6:24] * shang (~ShangWu@118.175.165.81) Quit (Quit: Ex-Chat)
[6:26] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[6:27] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[6:29] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[6:30] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[6:32] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[6:36] * rongze (~rongze@117.79.232.197) Quit (Ping timeout: 480 seconds)
[6:36] * sarob_ (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[6:36] * sarob_ (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[6:36] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[6:36] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) has joined #ceph
[6:38] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[6:39] * mwarwick (~mwarwick@203-219-79-122.static.tpgi.com.au) has joined #ceph
[6:44] * sarob (~sarob@nat-dip3.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[6:55] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[6:59] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[6:59] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[7:03] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[7:05] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Read error: Operation timed out)
[7:07] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[7:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[7:15] * sleinen (~Adium@2001:620:0:26:5040:fdda:cffb:f931) has joined #ceph
[7:18] * mwarwick (~mwarwick@203-219-79-122.static.tpgi.com.au) Quit (Ping timeout: 480 seconds)
[7:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[7:25] * shimo (~A13032@122x212x216x66.ap122.ftth.ucom.ne.jp) Quit (Read error: Connection reset by peer)
[7:26] * sleinen1 (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[7:28] * sleinen2 (~Adium@2001:620:0:26:341d:15d6:bb7a:62cd) has joined #ceph
[7:29] * sleinen (~Adium@2001:620:0:26:5040:fdda:cffb:f931) Quit (Ping timeout: 480 seconds)
[7:34] * sleinen1 (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[7:37] * rongze (~rongze@123.151.28.73) has joined #ceph
[7:37] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[7:39] * sleinen2 (~Adium@2001:620:0:26:341d:15d6:bb7a:62cd) Quit (Quit: Leaving.)
[7:43] * WarrenUsui (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) Quit (Read error: Connection reset by peer)
[7:44] * wusui1 (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) has joined #ceph
[7:46] * WarrenUsui1 (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) has joined #ceph
[7:51] * aardvark1 (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) Quit (Ping timeout: 480 seconds)
[7:52] * topro (~prousa@host-62-245-142-50.customer.m-online.net) Quit (Quit: Konversation terminated!)
[7:52] * topro (~prousa@host-62-245-142-50.customer.m-online.net) has joined #ceph
[7:53] * wusui1 (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) Quit (Ping timeout: 480 seconds)
[7:54] <symmcom> Hello all, i created a MON manually according to the doc but it kept sayiing MON down. Any way i can figure out what causing it to be down even after the host restart?
[7:54] * WarrenUsui (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) has joined #ceph
[7:59] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) Quit (Quit: Leaving.)
[8:02] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:09] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[8:10] * shig_ (~davidb@faith.oztechninja.com) has left #ceph
[8:14] * KindTwo (KindOne@h178.52.186.173.dynamic.ip.windstream.net) has joined #ceph
[8:15] <topro> symmcom: is the new mon process on that host running?
[8:15] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[8:15] * KindTwo is now known as KindOne
[8:16] <symmcom> topro: sorry fairly new in CEPH. how do i check the process ?
[8:16] <symmcom> i mean what is the process name
[8:17] <topro> process name (at least on linux) is something like ceph-mon
[8:18] <symmcom> topro: ps aux | grep ceph-mon shows nothing
[8:18] <topro> on linux something like "ps aux | grep ceph" should give you a list of all running ceph processes
[8:19] <topro> you created that cluster using ceph-deploy?
[8:20] <symmcom> yes, cluster was created over 3 months ago. still using the cluster. i just wanted to add another new MON cause i am taking one down due to old hardware
[8:21] <topro> i'm not familiar with how ceph-deployed cluster work. do you have a /etc/ceph/ceph.conf on all machines?
[8:21] <symmcom> i have 5 mon running since cluster creation. this 6th new one having issue. yes ceph.conf is verified on all nodes
[8:22] <symmcom> what do i do to start the process manually ?
[8:22] <topro> you added the new mon to ceph.conf and mad this identical ceph.conf available to all nodes?
[8:23] <symmcom> i didnt use ceph-deploy to create this new MON though. i followed this doc http://ceph.com/docs/master/rados/operations/add-or-rm-mons/ to add mon manually
[8:23] <topro> is your os using init.d (debian)?
[8:24] <symmcom> Ubuntu
[8:24] <topro> i was afraid you would say so. I'm not familiar with ubuntu upstart :/ could you figure out how to restart ceph service on that single new mon?
[8:24] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[8:25] <topro> and see what console output you get
[8:25] <topro> or have a look at /var/log/ceph/ceph-mon* on that node
[8:26] <topro> I assume the proble is somewhere in the range of keyring/auth settings of that new node
[8:26] <symmcom> according to doc, #service ceph -a restart mon should restart/start a mon, but i get error saying the host is not defined in ceph.conf
[8:27] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:27] <topro> I would not use "-a" as that would try to restart all mons of you cluster, across all nodes AFAIK
[8:29] <symmcom> this is what i am getting http://pastebin.com/mX9EGhwJ
[8:32] <symmcom> topro: one thing i m noticing, this new mon dont have this file, ceph-mon.ceph-mon-03.asok inside /var/run/ceph/ like all other 5 mon does
[8:34] * aarontc (~aaron@static-50-126-79-226.hlbo.or.frontiernet.net) Quit (Quit: Bye...)
[8:36] * shimo (~A13032@122x212x216x66.ap122.ftth.ucom.ne.jp) has joined #ceph
[8:38] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[8:38] * aarontc (~aaron@static-50-126-79-226.hlbo.or.frontiernet.net) has joined #ceph
[8:40] <topro> symmcom: could you past you ceph.conf?
[8:40] <topro> paste
[8:41] * madkiss (~madkiss@2001:6f8:12c3:f00f:956b:85b3:bb5:e176) Quit (Quit: Leaving.)
[8:42] * rendar (~s@host254-179-dynamic.1-87-r.retail.telecomitalia.it) has joined #ceph
[8:42] <symmcom> topro: http://pastebin.com/irRvZCWH . ceph-mon-03 is the new one i am trying connect
[8:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[8:44] <topro> symmcom: this is your complete ceph.conf?
[8:45] <symmcom> Yes, except i X'ed out the fsid
[8:45] <symmcom> thats all i got in my ceph.conf
[8:46] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[8:46] <topro> i think you need someone with ceph-deploy experience. my manually set up cluster seems to work completely different
[8:46] <symmcom> ceph -s
[8:46] <symmcom> this still shows the new mon down
[8:47] <topro> how about "ceph auth list"
[8:48] * mwarwick (~mwarwick@110-174-133-236.static.tpgi.com.au) has joined #ceph
[8:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:55] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[9:02] * mschiff (~mschiff@tmo-109-53.customers.d1-online.com) has joined #ceph
[9:04] <symmcom> topro: if u r still here, thank u very much for trying to help. i think it is working now. Just for the heck of it i wanted to see if it was firewall issue so i ran #sudo ufw allow 6789 then tried to readd the mon to the cluster and worked!
[9:05] <topro> ok, cool
[9:07] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[9:09] * fouxm (~fouxm@2a04:2500:0:b00:a8d6:853a:7aad:702c) has joined #ceph
[9:10] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[9:13] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[9:30] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[9:32] * aarontc (~aaron@static-50-126-79-226.hlbo.or.frontiernet.net) has left #ceph
[9:42] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[9:42] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[9:42] * ChanServ sets mode +v andreask
[9:43] * madkiss (~madkiss@2001:6f8:12c3:f00f:a1fa:3728:369a:a982) has joined #ceph
[9:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[9:48] * ScOut3R (~ScOut3R@212.96.46.212) has joined #ceph
[9:55] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[10:07] * mwarwick (~mwarwick@110-174-133-236.static.tpgi.com.au) Quit (Quit: Leaving.)
[10:08] * yanzheng (~zhyan@jfdmzpr02-ext.jf.intel.com) Quit (Remote host closed the connection)
[10:10] * aarontc (~aarontc@aarontc-1-pt.tunnel.tserv14.sea1.ipv6.he.net) has joined #ceph
[10:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[10:13] * kainz (~anders@emp.sdu.dk) Quit (Quit: Ex-Chat)
[10:17] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[10:18] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) has joined #ceph
[10:21] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Read error: Operation timed out)
[10:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[10:24] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[10:24] * mattt_ (~textual@92.52.76.140) has joined #ceph
[10:24] * andreask (~andreask@zid-vpnn061.uibk.ac.at) has joined #ceph
[10:24] * ChanServ sets mode +v andreask
[10:32] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[10:32] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[10:40] * Siva_ (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[10:42] * allsystemsarego (~allsystem@5-12-240-115.residential.rdsnet.ro) has joined #ceph
[10:43] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[10:43] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Read error: Operation timed out)
[10:43] * Siva_ is now known as Siva
[10:54] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) has joined #ceph
[10:57] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[11:00] * TMM (~hp@194.78.35.195) has joined #ceph
[11:10] * xdeller (~xdeller@91.218.144.129) has joined #ceph
[11:13] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:22] * mnash (~chatzilla@vpn.expressionanalysis.com) Quit (Read error: Operation timed out)
[11:23] * TMM (~hp@194.78.35.195) Quit (Ping timeout: 480 seconds)
[11:24] * simulx (~simulx@vpn.expressionanalysis.com) Quit (Ping timeout: 480 seconds)
[11:25] * yanzheng (~zhyan@134.134.139.74) has joined #ceph
[11:28] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[11:29] * i_m (~ivan.miro@deibp9eh1--blueice2n2.emea.ibm.com) has joined #ceph
[11:30] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[11:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:48] * dosaboy (~dosaboy@65.93.189.91.lcy-01.canonistack.canonical.com) Quit (Quit: leaving)
[11:49] * dosaboy (~dosaboy@65.93.189.91.lcy-01.canonistack.canonical.com) has joined #ceph
[11:50] * mxmln (~mxmln@tmo-106-25.customers.d1-online.com) has joined #ceph
[11:53] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:03] * simulx (~simulx@66-194-114-178.static.twtelecom.net) has joined #ceph
[12:12] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[12:13] * laithshadeed (~lss@80.227.44.122) Quit (Ping timeout: 480 seconds)
[12:13] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[12:20] * dxd828 (~dxd828@195.191.107.205) has joined #ceph
[12:20] * mxmln (~mxmln@tmo-106-25.customers.d1-online.com) Quit (Read error: Connection reset by peer)
[12:22] * dxd828 (~dxd828@195.191.107.205) Quit ()
[12:28] * dxd828 (~dxd828@195.191.107.205) has joined #ceph
[12:32] * dxd828 (~dxd828@195.191.107.205) Quit ()
[12:36] * laithshadeed (~lss@80.227.44.122) has joined #ceph
[12:36] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[12:40] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[12:40] * mschiff (~mschiff@tmo-109-53.customers.d1-online.com) Quit (Read error: No route to host)
[12:40] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[12:44] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[12:46] * mxmln (~mxmln@tmo-106-25.customers.d1-online.com) has joined #ceph
[12:53] * masterpe_ (~masterpe@2a01:670:400::43) has joined #ceph
[12:54] * masterpe (~masterpe@2a01:670:400::43) Quit (Ping timeout: 480 seconds)
[12:57] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:59] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[13:09] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[13:09] * swinchen (~swinchen@samuel-winchenbach.ums.maine.edu) Quit (Ping timeout: 480 seconds)
[13:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:15] * madkiss (~madkiss@2001:6f8:12c3:f00f:a1fa:3728:369a:a982) Quit (Ping timeout: 480 seconds)
[13:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:25] * mnash (~chatzilla@vpn.expressionanalysis.com) has joined #ceph
[13:34] * alfredodeza (~alfredode@c-24-131-46-23.hsd1.ga.comcast.net) has joined #ceph
[13:37] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) has joined #ceph
[13:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:53] * mxmln (~mxmln@tmo-106-25.customers.d1-online.com) Quit (Ping timeout: 480 seconds)
[13:53] * ksingh (~Adium@2001:708:10:10:c93a:6aa4:fcbe:d158) has joined #ceph
[13:53] <ksingh> GUYS PLEASE help
[13:53] <ksingh> i am trying to connect openstack cinder and ceph
[13:54] <ksingh> in cinder volume logs i am getting this eerror
[13:54] <ksingh> Bad or unexpected response from the storage volume backend API: error connecting to ceph cluster
[13:59] <leseb> loicd: 14:20 ??? 14:50 ?? Ceph: de facto storage backend for OpenStack ?? by Sebastien Han ??? Cloud Engineer - eNovance
[13:59] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) Quit (Quit: Ex-Chat)
[13:59] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:01] * piti (~piti@82.246.190.142) Quit (Ping timeout: 480 seconds)
[14:04] * e1mer (~erivera@121.54.77.87) has joined #ceph
[14:05] <baffle> When I create an OSD filesystem structure with "ceph-osd -i <osd id from config> --mkfs" ; Should I also use --mkkey? I don't really find much documentation about what --mkkey does, but I see that it tries to write to /etc/ceph/keyring and failing.. :)
[14:06] <baffle> Is that the recommended way to ready an OSD manually?
[14:07] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[14:13] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:19] * tsnider (~tsnider@nat-216-240-30-23.netapp.com) has joined #ceph
[14:23] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[14:24] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:31] * madkiss (~madkiss@089144194119.atnat0003.highway.a1.net) has joined #ceph
[14:34] * joshuay04 (~joshuay04@rrcs-74-218-204-10.central.biz.rr.com) has joined #ceph
[14:34] <joshuay04> Quick question, my data center is loosing power. I have a cluster of 5 nodes, what is the proper way to protect the cluster from power failure? Do I just shut the nodes off 1 at a time or do I have to do anything special?
[14:35] * madkiss (~madkiss@089144194119.atnat0003.highway.a1.net) Quit ()
[14:36] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) Quit (Quit: Ex-Chat)
[14:38] * kl4m (~kl4m@66.254.36.166) has joined #ceph
[14:39] * e1mer (~erivera@121.54.77.87) Quit (Quit: This computer has gone to sleep)
[14:43] <joao> afaik you can shutdown the whole thing at once, just stop the daemons gracefully
[14:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:48] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[14:50] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:56] * sleinen1 (~Adium@2001:620:0:26:88e:162f:a7f:a192) has joined #ceph
[15:01] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[15:04] * markbby (~Adium@168.94.245.3) has joined #ceph
[15:06] * glzhao (~glzhao@118.195.65.67) has joined #ceph
[15:08] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[15:11] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[15:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:15] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) has joined #ceph
[15:16] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[15:21] * swinchen (~swinchen@samuel-winchenbach.ums.maine.edu) has joined #ceph
[15:21] * joshuay04 (~joshuay04@rrcs-74-218-204-10.central.biz.rr.com) Quit ()
[15:24] * ksingh (~Adium@2001:708:10:10:c93a:6aa4:fcbe:d158) has left #ceph
[15:26] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[15:28] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:29] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[15:34] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) Quit (Quit: Ex-Chat)
[15:35] * mnash (~chatzilla@vpn.expressionanalysis.com) Quit (Quit: ChatZilla 0.9.90.1 [Firefox 20.0.1/20130409194949])
[15:38] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[15:38] * mschiff (~mschiff@tmo-109-53.customers.d1-online.com) has joined #ceph
[15:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:44] * mnash (~chatzilla@66-194-114-178.static.twtelecom.net) has joined #ceph
[15:50] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[15:51] * yanzheng (~zhyan@134.134.139.74) Quit (Remote host closed the connection)
[15:52] * mschiff (~mschiff@tmo-109-53.customers.d1-online.com) Quit (Ping timeout: 480 seconds)
[15:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:57] * Siva (~sivat@117.192.37.236) has joined #ceph
[16:00] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[16:05] * Siva (~sivat@117.192.37.236) Quit (Ping timeout: 480 seconds)
[16:09] * Siva (~sivat@115.241.12.216) has joined #ceph
[16:12] * Siva_ (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[16:12] * japuzzo (~japuzzo@ool-4570886e.dyn.optonline.net) has joined #ceph
[16:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[16:17] * Siva (~sivat@115.241.12.216) Quit (Ping timeout: 480 seconds)
[16:17] * Siva_ is now known as Siva
[16:18] * cfreak201 (~cfreak200@p4FF3E351.dip0.t-ipconnect.de) has joined #ceph
[16:19] * dxd828 (~dxd828@195.191.107.205) has joined #ceph
[16:21] * terje (~root@135.109.216.239) has joined #ceph
[16:21] * cfreak200 (~cfreak200@p4FF3F14F.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[16:27] <japuzzo> Off topic but, this is a nice break from distributed FS work https://www.google.co.uk/ the doctor who game!
[16:28] * gregmark (~Adium@68.87.42.115) Quit (Quit: Leaving.)
[16:28] * gregmark (~Adium@cet-nat-254.ndceast.pa.bo.comcast.net) has joined #ceph
[16:31] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[16:36] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) has joined #ceph
[16:36] * ChanServ sets mode +o scuttlemonkey
[16:38] <terje> Hi, I have an OSD that's almost at capacity.
[16:38] <terje> I'd like to have ceph rebalance it
[16:39] <terje> my one OSD is at 95% capacity and the rest are ~25%
[16:39] <terje> is there some way to force that particular OSD to start moving PG's off?
[16:41] <topro> terje? what ceph version?
[16:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[16:44] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) has joined #ceph
[16:45] * rongze (~rongze@123.151.28.73) Quit (Remote host closed the connection)
[16:49] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[16:52] * Machske (~Bram@d5152D87C.static.telenet.be) Quit (Ping timeout: 480 seconds)
[16:52] * andreask (~andreask@zid-vpnn061.uibk.ac.at) Quit (Read error: Connection reset by peer)
[16:53] <terje> ceph-0.61.8-35.g26b1e97
[16:57] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[16:59] * thomnico (~thomnico@2a01:e35:8b41:120:4c6c:3b8:1f70:eaa8) has joined #ceph
[17:01] * mtanski_ (~mtanski@69.193.178.202) has joined #ceph
[17:02] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[17:03] <mikedawson> terje: how many osds, and how many PGs?
[17:03] * mtanski_ (~mtanski@69.193.178.202) Quit ()
[17:04] * rongze (~rongze@211.155.113.166) has joined #ceph
[17:05] <mikedawson> terje: there is a 'ceph osd reweight-by-utilization' command, but I suspect too few Placement Groups is your root problem
[17:05] * mtanski_ (~mtanski@69.193.178.202) has joined #ceph
[17:06] * mtanski (~mtanski@69.193.178.202) Quit (Read error: Connection reset by peer)
[17:06] * mtanski_ is now known as mtanski
[17:11] * dxd828 (~dxd828@195.191.107.205) Quit (Quit: Computer has gone to sleep.)
[17:11] <topro> terje: with 0.61 ceph -s won't give you a warning if you have too few PGs per OSD (like what mikedawson told you). could you paste output of ceph -s somewhere?
[17:11] * dxd828 (~dxd828@195.191.107.205) has joined #ceph
[17:12] <mancdaz> terje: if you've increased the pg_num for the pool also make sure you increase the pgp_num
[17:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[17:12] <mancdaz> else no data movement will take place
[17:16] * BillK (~BillK-OFT@124-169-96-192.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[17:17] * BillK (~BillK-OFT@58-7-102-11.dyn.iinet.net.au) has joined #ceph
[17:22] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[17:22] * BillK (~BillK-OFT@58-7-102-11.dyn.iinet.net.au) Quit (Read error: Connection reset by peer)
[17:25] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[17:25] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[17:27] * dxd828 (~dxd828@195.191.107.205) Quit (Quit: Computer has gone to sleep.)
[17:30] * BillK (~BillK-OFT@124-148-85-36.dyn.iinet.net.au) has joined #ceph
[17:35] * dxd828 (~dxd828@195.191.107.205) has joined #ceph
[17:36] * rongze (~rongze@211.155.113.166) Quit (Ping timeout: 480 seconds)
[17:38] * sleinen (~Adium@user-28-18.vpn.switch.ch) has joined #ceph
[17:40] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[17:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[17:43] * dxd828 (~dxd828@195.191.107.205) Quit (Ping timeout: 480 seconds)
[17:45] * sleinen1 (~Adium@2001:620:0:26:88e:162f:a7f:a192) Quit (Ping timeout: 480 seconds)
[17:45] * `Kevin (~kevin@sys0.chi0.birchbox.com) Quit (Ping timeout: 480 seconds)
[17:48] * sleinen1 (~Adium@2001:620:0:26:6451:ff86:f36:fee7) has joined #ceph
[17:53] * nwat (~textual@eduroam-239-76.ucsc.edu) has joined #ceph
[17:54] * sleinen (~Adium@user-28-18.vpn.switch.ch) Quit (Ping timeout: 480 seconds)
[17:55] * JoeGruher (~JoeGruher@134.134.139.72) Quit (Remote host closed the connection)
[17:57] * clayb (~kvirc@69.191.241.59) has joined #ceph
[18:00] * mattch (~mattch@pcw3047.see.ed.ac.uk) Quit (Quit: Leaving.)
[18:01] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:03] * zackc (~zackc@0001ba60.user.oftc.net) Quit (Ping timeout: 480 seconds)
[18:04] * aliguori (~anthony@74.202.210.82) has joined #ceph
[18:06] * ScOut3R (~ScOut3R@212.96.46.212) Quit (Ping timeout: 480 seconds)
[18:07] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[18:07] * ChanServ sets mode +v andreask
[18:07] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[18:08] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) Quit (Quit: Leaving.)
[18:13] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:15] <terje> sorry for the delay - phone call..
[18:15] <terje> http://fpaste.org/56093/13851404/
[18:15] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[18:17] * i_m (~ivan.miro@deibp9eh1--blueice2n2.emea.ibm.com) Quit (Quit: Leaving.)
[18:18] <mikedawson> terje: which osd is nearly full?
[18:19] * sleinen1 (~Adium@2001:620:0:26:6451:ff86:f36:fee7) Quit (Quit: Leaving.)
[18:19] * sleinen (~Adium@130.59.94.210) has joined #ceph
[18:21] <terje> osd.14
[18:22] <terje> I should mention that those drives (where the OSD's live) vary in size.
[18:23] <terje> some are 3TB, some 2TB and this one is 1TB.
[18:23] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[18:23] <terje> I'll paste a df
[18:24] <terje> added df output: http://fpaste.org/56100/14107313/
[18:24] * thomnico (~thomnico@2a01:e35:8b41:120:4c6c:3b8:1f70:eaa8) Quit (Quit: Ex-Chat)
[18:26] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:26] <mikedawson> terje: if the drives vary in size, you would weight them accordingly.
[18:27] * sleinen (~Adium@130.59.94.210) Quit (Ping timeout: 480 seconds)
[18:28] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[18:28] * i_m (~ivan.miro@deibp9eh1--blueice1n1.emea.ibm.com) has joined #ceph
[18:30] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[18:34] * ircolle (~Adium@2601:1:8380:2d9:8d67:a3fe:f259:cc67) has joined #ceph
[18:36] <terje> I thought ceph just figured that out
[18:36] * i_m (~ivan.miro@deibp9eh1--blueice1n1.emea.ibm.com) Quit (Ping timeout: 480 seconds)
[18:36] <terje> I'll look into weighting, thanks.
[18:38] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[18:39] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[18:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:47] * DarkAce-Z (~BillyMays@50.107.53.200) has joined #ceph
[18:50] * mtanski (~mtanski@69.193.178.202) Quit (Quit: mtanski)
[18:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:52] * DarkAceZ (~BillyMays@50.107.53.200) Quit (Ping timeout: 480 seconds)
[18:53] * DarkAce-Z is now known as DarkAceZ
[18:55] * JoeGruher (~JoeGruher@134.134.139.72) has joined #ceph
[18:56] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[18:57] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[18:57] * rongze (~rongze@211.155.113.166) has joined #ceph
[19:02] * tsnider (~tsnider@nat-216-240-30-23.netapp.com) Quit (Ping timeout: 480 seconds)
[19:02] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[19:03] * alram (~alram@38.122.20.226) has joined #ceph
[19:04] * sjusthm (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) has joined #ceph
[19:06] * glzhao (~glzhao@118.195.65.67) Quit (Quit: leaving)
[19:07] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[19:07] * julian (~julian@125.70.134.54) Quit (Quit: Leaving)
[19:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[19:16] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Read error: Connection reset by peer)
[19:16] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[19:17] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[19:18] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[19:19] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[19:20] * sleinen1 (~Adium@2001:620:0:25:dc9a:6f7e:c193:cc71) has joined #ceph
[19:20] * mattt_ (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[19:22] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[19:24] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Ping timeout: 480 seconds)
[19:27] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[19:28] * dmsimard (~Adium@108.163.152.2) Quit (Remote host closed the connection)
[19:28] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[19:28] * dmsimard (~Adium@108.163.152.2) Quit (Remote host closed the connection)
[19:29] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[19:29] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[19:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[19:42] * rongze (~rongze@211.155.113.166) Quit (Remote host closed the connection)
[19:44] * sleinen1 (~Adium@2001:620:0:25:dc9a:6f7e:c193:cc71) Quit (Quit: Leaving.)
[19:45] * fouxm (~fouxm@2a04:2500:0:b00:a8d6:853a:7aad:702c) Quit (Remote host closed the connection)
[19:49] * bcat (~bcat@64-79-127-98.static.wiline.com) has joined #ceph
[19:50] * Guest6261 (wmat@wallace.mixdown.ca) has joined #ceph
[19:51] * Guest6261 is now known as wmat
[19:51] <bcat> hihi, morning guys
[19:51] <gkoch> Hi all. I am trying to get my first ceph cluster up. I have 3 mons running, and 1 osd. Trying to add a second osd on a different machine and I am getting this error: "** ERROR: unable to open OSD superblock on /var/lib/ceph/osd/osd.1/data: (2) No such file or directory." Not sure why it says no such file or directory - the directory exists and the xfs partition is mounted there. Any ideas?
[19:52] <JoeGruher> does ceph read-cache data? for example, if i have 100GB of RAM on my OSD servers and I'm using a 20GB data set for a read workload am I eventually going to be reading it out of memory or will ceph fetch from disk on every read? setting up some performance testing and wondering how large a data set i'm going to have to use to avoid excessive cache hits skewing the data.
[19:55] <bcat> hi, does anyone know how to set journal queue max ops and bytes properly, I have 60GB ssd serves for 4 OSD (4x14GB partitions)
[19:57] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[19:57] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Remote host closed the connection)
[19:57] * sarob (~sarob@nat-dip6.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:03] * dxd828 (~dxd828@212.183.140.59) has joined #ceph
[20:03] * sarob (~sarob@nat-dip6.cfw-a-gci.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[20:03] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[20:03] * zackc (~zackc@0001ba60.user.oftc.net) has joined #ceph
[20:05] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Remote host closed the connection)
[20:05] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[20:09] * tsnider (~tsnider@ip68-102-128-87.ks.ok.cox.net) has joined #ceph
[20:10] * xmltok (~xmltok@216.103.134.250) has joined #ceph
[20:11] * dxd828 (~dxd828@212.183.140.59) Quit (Ping timeout: 480 seconds)
[20:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:13] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Ping timeout: 480 seconds)
[20:19] * Cube (~Cube@12.248.40.138) has joined #ceph
[20:20] * nwat (~textual@eduroam-239-76.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[20:21] * tsnider1 (~tsnider@ip68-102-128-87.ks.ok.cox.net) has joined #ceph
[20:22] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:22] * Pedras (~Adium@216.207.42.132) has joined #ceph
[20:23] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[20:23] <dmsimard> xarses: god damnit.
[20:23] <dmsimard> xarses: I thought I had something working, then I get the fastcgi 411 error from radosgw
[20:26] * tsnider (~tsnider@ip68-102-128-87.ks.ok.cox.net) Quit (Ping timeout: 480 seconds)
[20:26] * tsnider (~tsnider@198.95.226.40) has joined #ceph
[20:26] <xarses> dmsimard: owich
[20:26] * dxd828 (~dxd828@85.255.232.211) has joined #ceph
[20:27] <dmsimard> xarses: qu?? ??? :p
[20:27] <xarses> dmsimard: i don't recall getting 411
[20:27] <dmsimard> xarses: It's the bug you told me about, that needs fastcgi from the ceph repo
[20:28] <dmsimard> Recoverable error: Object PUT failed: http:// 411 Length Required [first 60 chars of response] <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
[20:28] <xarses> where you able to find it?
[20:28] <dmsimard> Yeah, patching now and trying again
[20:28] <janos> html 2.0? haha
[20:28] <janos> wow
[20:29] <xarses> dmsimard: there are built packages on gitbuilder
[20:29] <xarses> http://gitbuilder.ceph.com/
[20:29] <xarses> dmsimard: it's a magical place full of all kinds of wonder
[20:29] * tsnider1 (~tsnider@ip68-102-128-87.ks.ok.cox.net) Quit (Ping timeout: 480 seconds)
[20:30] <dmsimard> Yeah, I got the repo thanks to https://github.com/xarses/puppet-ceph/commit/08c7a065604ecc637c61aad31f461e74086233cd :p
[20:33] * fouxm (~fouxm@5-49-192-225.hfc.dyn.abo.bbox.fr) has joined #ceph
[20:35] * dxd828 (~dxd828@85.255.232.211) Quit (Ping timeout: 480 seconds)
[20:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:46] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[20:50] <symmcom> Hello all,
[20:50] <symmcom> How can i auto mount CephFS during boon on Ubuntu
[20:50] <symmcom> *boot
[20:52] <symmcom> currently i m manually mounting after reboot using this: #ceph-fuse /mnt/XXXXXX/ -o nonempty
[20:53] * davidzlap (~Adium@ip68-5-239-214.oc.oc.cox.net) Quit (Quit: Leaving.)
[20:54] <dmsimard> symmcom: Use an entry in your fstab ?
[20:55] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[20:56] * JoeGruher (~JoeGruher@134.134.139.72) Quit (Ping timeout: 480 seconds)
[21:00] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:00] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) has joined #ceph
[21:01] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit ()
[21:02] * fouxm (~fouxm@5-49-192-225.hfc.dyn.abo.bbox.fr) Quit (Remote host closed the connection)
[21:03] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[21:04] * mtl1 (~Adium@c-67-176-54-246.hsd1.co.comcast.net) has joined #ceph
[21:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[21:14] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[21:26] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:29] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Remote host closed the connection)
[21:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[21:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:52] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[21:53] * sarob (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Remote host closed the connection)
[21:53] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[21:55] * sarob_ (~sarob@ip-64-134-225-149.public.wayport.net) has joined #ceph
[21:57] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[21:59] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) Quit (Quit: Leaving.)
[21:59] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Read error: Connection reset by peer)
[22:00] * sleinen1 (~Adium@2001:620:0:25:dc31:2585:3f5e:717c) has joined #ceph
[22:05] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[22:09] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[22:11] * yanzheng (~zhyan@134.134.139.76) has joined #ceph
[22:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:14] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has left #ceph
[22:17] * gkoch (~gkoch@38.86.161.178) Quit (Ping timeout: 480 seconds)
[22:27] * mtl1 (~Adium@c-67-176-54-246.hsd1.co.comcast.net) has left #ceph
[22:31] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:33] <loicd> leseb: ok. Would you be so kind as to ping me when https://www.eventbrite.com/e/openstack-in-action-4-tickets-7645801799 is updated ?
[22:37] * sleinen1 (~Adium@2001:620:0:25:dc31:2585:3f5e:717c) Quit (Quit: Leaving.)
[22:37] * john_barbee (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[22:37] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[22:37] * alfredodeza (~alfredode@c-24-131-46-23.hsd1.ga.comcast.net) has left #ceph
[22:38] * japuzzo (~japuzzo@ool-4570886e.dyn.optonline.net) Quit (Quit: Leaving)
[22:39] * john_barbee (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit ()
[22:39] * john_barbee (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[22:41] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[22:41] * allsystemsarego (~allsystem@5-12-240-115.residential.rdsnet.ro) Quit (Quit: Leaving)
[22:41] * linuxkidd (~linuxkidd@ip72-193-217-254.lv.lv.cox.net) has joined #ceph
[22:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:45] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[22:53] * yanzheng (~zhyan@134.134.139.76) Quit (Ping timeout: 480 seconds)
[22:56] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:57] * simulx2 (~simulx@vpn.expressionanalysis.com) has joined #ceph
[22:57] * symmcom (~symmcom@184.70.203.22) Quit (Read error: Connection reset by peer)
[23:00] * simulx (~simulx@66-194-114-178.static.twtelecom.net) Quit (Ping timeout: 480 seconds)
[23:01] * gkoch (~gkoch@38.86.161.178) has joined #ceph
[23:02] <gkoch> Hi all trying to start radosgw on a new cluster. no output when I start it via init.d, status says not running, and nothing shows in the logs. Any ideas? I was following http://ceph.com/docs/master/man/8/radosgw/
[23:05] * kl4m (~kl4m@66.254.36.166) Quit (Remote host closed the connection)
[23:07] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[23:11] * xarses (~andreww@64-79-127-122.static.wiline.com) Quit (Ping timeout: 480 seconds)
[23:14] * Shmouel (~Sam@fny94-12-83-157-27-95.fbx.proxad.net) Quit (Read error: Connection reset by peer)
[23:20] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[23:22] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:23] * angdraug (~angdraug@64-79-127-122.static.wiline.com) has joined #ceph
[23:26] * dmsimard (~Adium@108.163.152.2) Quit (Quit: Leaving.)
[23:27] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[23:31] * dmsimard (~Adium@70.38.0.248) has joined #ceph
[23:38] * bcat (~bcat@64-79-127-98.static.wiline.com) Quit ()
[23:41] * clayb (~kvirc@69.191.241.59) Quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/)
[23:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:46] * xmltok (~xmltok@216.103.134.250) Quit (Quit: Leaving...)
[23:47] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[23:47] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:49] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[23:50] * markbby (~Adium@168.94.245.3) Quit (Quit: Leaving.)
[23:50] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) has joined #ceph
[23:51] * sleinen1 (~Adium@2001:620:0:26:1167:edbb:fd1:6af) has joined #ceph
[23:51] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Ping timeout: 480 seconds)
[23:52] * xarses (~andreww@64-79-127-122.static.wiline.com) Quit (Ping timeout: 480 seconds)
[23:53] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[23:53] * ChanServ sets mode +o joao|lap
[23:57] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.