#ceph IRC Log

Index

IRC Log for 2013-06-01

Timestamps are in GMT/BST.

[0:02] <Tamil> loicd: will let you know if there is any failure from your code change, i have just started looking into the nightly failures
[0:04] <loicd> Tamil: it looks like none of the failures are related to my changes, which is a great relief :-) I'll get a good night sleep. Thanks for your help :-D
[0:05] <Tamil> loicd: :)
[0:13] * BillK (~BillK@124-148-124-185.dyn.iinet.net.au) has joined #ceph
[0:15] * terje_ (~joey@63-154-132-52.mpls.qwest.net) has joined #ceph
[0:17] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[0:17] <Kioob> sjust : force_create_pg to fix the fact that PG are stuck in "creating" state ? But... it's after using this command that I have that...
[0:18] <sjust> oh
[0:18] <sjust> that command needs to be fixed then
[0:18] <sjust> probably
[0:18] <sjust> one moment
[0:18] * terje- (~root@135.109.216.239) Quit (Quit: Lost terminal)
[0:20] * amb (~amb@82-69-2-201.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[0:20] <sjust> odd
[0:20] <sjust> can you file a bug?
[0:20] <Kioob> First problem was that I have "imcomplete" PG (I loose 2 OSD on a 2 replica scheme) ; and can't make the cluster recover a "HEALTH_OK" state. So I tried the "force_create_pg"
[0:20] <Kioob> Yes I can
[0:23] * terje_ (~joey@63-154-132-52.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[0:24] <Kioob> sjust: I need to create a new bug, or comment the issue 4813 ?
[0:24] <sjust> did you create a pool immediately prior to the stuck creating pgs?
[0:25] <Kioob> no
[0:25] <sjust> did you split pgs?
[0:25] <Kioob> no
[0:25] <sjust> it's a new bug
[0:25] <Kioob> ok
[0:25] <sjust> did you force_create_pg on an incomplete pg?
[0:25] <Kioob> yes
[0:26] <sjust> oh, that won't work
[0:26] <sjust> I'm not sure what that would do
[0:27] <sjust> I'd create the bug anyway
[0:27] <Kioob> well, I see it doesn't work :p
[0:27] <Kioob> ok thanks
[0:27] <Kioob> and... do you know what is the true solution to fix incomplete PG ? I didn't find any information about that
[0:28] * diegows (~diegows@190.190.2.126) has joined #ceph
[0:29] * PerlStalker (~PerlStalk@72.166.192.70) Quit (Quit: ...)
[0:29] <sjust> do you have dead osds?
[0:29] <Kioob> I add yes, but I re-add it
[0:29] <Kioob> I had*
[0:33] <Kioob> mmm in fact. I had 2 dead OSD (so data loss & incomplete PG), then I replace one of them.
[0:33] <sjust> they were incomplete before you readded it?
[0:33] <Kioob> So one of them is still down & out
[0:33] <sjust> you might have to mark that one lost
[0:34] <Kioob> it's already mark as lost
[0:34] <sjust> oh
[0:34] <Kioob> (both was marked as lost)
[0:34] <sjust> ceph pg query on one of the incomplete ones?
[0:36] <Kioob> http://pastebin.com/5Q5ZHqn9
[0:37] <Kioob> it's a huge one
[0:37] <Kioob> Lost OSD was 19 and 25. And 19 was replaced and is running
[0:38] <nigwil> I've run into issue 4855 (which is marked as not able to reproduce): http://pastebin.com/FhmRfjbC
[0:42] * ScOut3R (~ScOut3R@540240A4.dsl.pool.telekom.hu) has joined #ceph
[0:42] <sjust> can you add to the bug filesystem, osd version (fea782543a844bb277ae94d3391788b76c5bee60), leveldb package version
[0:43] <mech422> if I just want everything in my CRUSH map evenly weighted, can I just leave the weights out of the conf file ?
[0:43] <sjust> Kioob: can you restart osd.19 with 'debug osd = 20' in the [osd] section of it's ceph.conf, wait for the cluster to stabilize, and attach the log to the bug?
[0:43] <sjust> also debug ms = 1
[0:43] <Kioob> ok, thanks
[0:43] * redeemed (~redeemed@static-71-170-33-24.dllstx.fios.verizon.net) has joined #ceph
[0:44] <nigwil> sjust: "can you add" was that to Kioob or me?
[0:44] <Kioob> nigwil: you I suppose ;)
[0:45] <nigwil> not sure whether I'd jumped into your bug triage :-)
[0:45] * ScOut3R (~ScOut3R@540240A4.dsl.pool.telekom.hu) Quit (Remote host closed the connection)
[0:45] * redeemed (~redeemed@static-71-170-33-24.dllstx.fios.verizon.net) Quit ()
[0:47] <sjust> nigwil: that was for you
[0:48] <sjust> if you can reproduce on a clean osd with
[0:48] <sjust> debug filestore = 20
[0:48] <sjust> debug journal = 20
[0:48] <sjust> debug ms = 1
[0:48] <sjust> debug osd = 20
[0:48] <sjust> that would be awesome
[0:48] <phantomcircuit> im on 0.53.3 and currently backfilling a new osd
[0:48] <sjust> also, can you attach the dmseg output from that node as well?
[0:48] <phantomcircuit> it's going pretty slowly much below what the device is capable of
[0:49] <phantomcircuit> i've already set osd_max_backfills to 100
[0:49] <phantomcircuit> what else might be throttling?
[0:49] * aliguori (~anthony@cpe-70-112-157-87.austin.res.rr.com) has joined #ceph
[0:49] <sjust> you can turn up osd_recovery_max_active to 30
[0:49] <sjust> phantomcircuit: it will impact client IO though
[0:50] <phantomcircuit> sjust, that's fine there's really not very much client io anyways
[0:52] <phantomcircuit> seems to be sitting at ~30MB/s
[0:54] <phantomcircuit> yeah it's at exactly 100 mbps
[0:54] <phantomcircuit> sjust, anything else? :)
[0:54] <sjust> no improvement?
[0:54] <Kioob> sjust: logs are big... 380MB, still growing
[0:55] <sjust> Kioob: you can stop once the cluster stabilizes
[0:55] <Kioob> ok
[0:55] <phantomcircuit> sjust, seems to have not changed at all
[0:55] <sjust> you injectargs'd?
[0:55] <phantomcircuit> and it really is at almost exactly 100 mbps
[0:55] <phantomcircuit> i cant imagine that's a coincidence
[0:55] <Kioob> sjust: no, I restart OSD.19
[0:55] <sjust> Kioob: sorry, meant phantomcircuit
[0:55] <phantomcircuit> sjust, { "success": "applying configuration change: osd_recovery_max_active = '30'\n"}
[0:56] <sjust> yah
[0:56] <Kioob> ok :)
[0:56] <sjust> try osd_recovery_max_chunk = 67108864
[0:56] <sjust> phantomcircuit: are they little files?
[0:56] <phantomcircuit> it's rbd volumes
[0:57] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[0:57] <phantomcircuit> so should be lots of 4MB objects
[0:57] <sjust> hmm
[0:57] <sjust> yeah
[0:57] <sjust> nvm on that last one then
[0:57] <phantomcircuit> 106.2 Mb/s
[0:57] <sjust> where are you seeing that?
[0:58] <phantomcircuit> bmon
[0:58] <phantomcircuit> there's nothing else running on this system (yet)
[0:58] <sjust> how many osds on the system?
[0:58] <phantomcircuit> only 3
[0:58] <phantomcircuit> 2 on the other system
[0:58] <phantomcircuit> 1 on this one
[0:59] <sjust> how are the journals set up?
[0:59] <sjust> oh, so 2 osds recovering to 1 new osd?
[0:59] <phantomcircuit> yes
[0:59] <sjust> how many pgs?
[0:59] <Kioob> sjust: not sure to follow : in witch bug report should I paste my logs ?
[0:59] <phantomcircuit> wait shoudl the osd_recovy_max_active been for the recovering osds?
[0:59] <sjust> Kioob: a new one
[0:59] <phantomcircuit> im thinking yes
[0:59] <sjust> phantomcircuit: for all of them
[0:59] <phantomcircuit> lol
[0:59] <phantomcircuit> ok one sec
[1:00] <Kioob> ok, about stuck PG in "creating" state, or PG which stay in "imcomplete" state ? :p
[1:00] <sjust> Kioob: same bug actually
[1:00] <sjust> you need to lay out the entire story
[1:00] <Kioob> ok, thanks :)
[1:00] <Kioob> I try to write that
[1:00] <Tamil> mech422: what do you mean by leave weights out of the conf file?
[1:01] <phantomcircuit> wat
[1:02] <sjust> phantomcircuit: hmm?
[1:02] <phantomcircuit> nvm
[1:02] <phantomcircuit> i had osd_recovery_max_active=30 in injectargs when it should be --osd_recovery_max_active 30
[1:03] <sjust> yeah
[1:04] <phantomcircuit> 60 MB/s
[1:04] <phantomcircuit> ok so it's about doubled now
[1:05] <sjust> try raising it to 50? (just to confirm that it's saturated)
[1:05] <sjust> how are the journals set up?
[1:06] <phantomcircuit> sjust, zfs filesystem volume with an ssd log device
[1:06] <phantomcircuit> easily does 5k write iops and 120 MB/s
[1:06] <sjust> phantomcircuit: whoa
[1:06] <sjust> fancy
[1:06] <phantomcircuit> yeah fancy up in here
[1:06] <sjust> but you have the journal on the same backing device as the osd?
[1:07] <phantomcircuit> literally system dies under the load without all the random writes from vms being absorbed by the ssd
[1:07] <phantomcircuit> sjust, yeah i do
[1:07] <sjust> ok, the backfilling osd is sustaining 60MB/s*2
[1:07] <sjust> which is about right?
[1:08] <phantomcircuit> yeah should be finished any minute
[1:08] <sjust> k, you probably want to play with that setting until you find the max value that saturates
[1:08] <sjust> though it shouldn't be excessively wasteful to use more I suppos
[1:08] <sjust> *suppose
[1:10] <phantomcircuit> recovering 16 o/s, 64654KB/s
[1:10] <phantomcircuit> that'll do
[1:10] <sjust> yup
[1:10] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[1:13] * dxd828_ (~dxd828@host-92-24-117-118.ppp.as43234.net) Quit (Quit: Textual IRC Client: www.textualapp.com)
[1:17] <phantomcircuit> sjust, recovering 15E o/s, 15EB/s
[1:17] <phantomcircuit> LOL
[1:17] <phantomcircuit> ok then i dont think that is exactly accurate
[1:22] <Kioob> sjust: is it ok http://tracker.ceph.com/issues/5226 ? Should I add other informations ?
[1:27] * jlogan (~Thunderbi@2600:c00:3010:1:1::40) Quit (Ping timeout: 480 seconds)
[1:30] <nigwil> having "lost" an OSD and the cluster recovered ok, and tree showed it as down, I want to re-create it. So I did ceph osd crush remove osd.10, now the cluster is in recovery again?
[1:36] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) has joined #ceph
[1:38] * ghartz (~ghartz@ill67-1-82-231-212-191.fbx.proxad.net) Quit (Read error: Connection reset by peer)
[1:47] <sjust> Kioob: looks good
[1:48] <sjust> nigwil: that probably changed the crush mapping slightly
[1:51] <Kioob> ok thanks sjust
[1:54] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[2:09] * mschiff_ (~mschiff@port-1469.pppoe.wtnet.de) has joined #ceph
[2:13] * tnt (~tnt@91.176.24.98) Quit (Ping timeout: 480 seconds)
[2:13] * Cube (~Cube@12.248.40.138) Quit (Ping timeout: 480 seconds)
[2:15] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[2:16] * mschiff (~mschiff@port-29027.pppoe.wtnet.de) Quit (Ping timeout: 480 seconds)
[2:17] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) Quit (Quit: bia)
[2:18] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) has joined #ceph
[2:20] * terje_ (~joey@63-154-146-229.mpls.qwest.net) has joined #ceph
[2:25] * LeaChim (~LeaChim@176.250.167.111) Quit (Ping timeout: 480 seconds)
[2:28] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) Quit (Quit: bia)
[2:28] * terje_ (~joey@63-154-146-229.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[2:29] * xinxinsh (~xinxinsh@134.134.137.73) has joined #ceph
[2:29] * dpippenger (~riven@206-169-78-213.static.twtelecom.net) Quit (Quit: Leaving.)
[2:45] * xinxinsh (~xinxinsh@134.134.137.73) Quit (Quit: Leaving)
[2:45] * xinxinsh (~xinxinsh@134.134.137.73) has joined #ceph
[2:46] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[2:48] <mech422> if you try to restart a monitor, and it doesn't want to work - is it safe to just 'recreate' it with ceph-deploy mon create ?
[2:57] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[3:06] <mech422> Hmm - it appears the answer is 'yes' :-)
[3:07] * rturk is now known as rturk-away
[3:07] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[3:14] * xinxinsh (~xinxinsh@134.134.137.73) Quit (Quit: Leaving)
[3:25] <Tamil> mech422: i have never tried it myself though, maybe give it a try
[3:25] <mech422> it seemed to work ok - at least on my empty cluster
[3:25] <mech422> my osd's don't restart on reboot either - I have to 'ceph-deploy osd activate ...' them
[3:26] <mech422> I think its because I just have a 'stub' ceph.conf on each machine ..
[3:27] <mech422> ceph-deploy conf files don't mention osd's or mons at all - I think the node just doesn't know what it's supposed to restart ?
[3:31] * terje_ (~joey@63-154-146-229.mpls.qwest.net) has joined #ceph
[3:39] * terje_ (~joey@63-154-146-229.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[3:41] * terje_ (~joey@63-154-136-95.mpls.qwest.net) has joined #ceph
[3:45] * terje-_ (~terje@63-154-136-95.mpls.qwest.net) has joined #ceph
[3:45] * Tamil (~tamil@38.122.20.226) Quit (Quit: Leaving.)
[3:49] * terje_ (~joey@63-154-136-95.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[3:50] * terje- (~terje@63-154-136-95.mpls.qwest.net) has joined #ceph
[3:53] * terje-_ (~terje@63-154-136-95.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[3:58] * terje- (~terje@63-154-136-95.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[4:16] * terje_ (~joey@63-154-136-95.mpls.qwest.net) has joined #ceph
[4:19] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[4:24] * terje_ (~joey@63-154-136-95.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[4:35] * terje-_ (~terje@63-154-136-95.mpls.qwest.net) has joined #ceph
[4:40] * noahmehl_ (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[4:41] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Read error: Operation timed out)
[4:41] * noahmehl_ is now known as noahmehl
[4:43] * terje-_ (~terje@63-154-136-95.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[4:44] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[4:51] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[5:06] * Vanony (~vovo@i59F7A407.versanet.de) has joined #ceph
[5:10] * loicd (~loic@2a01:e35:2eba:db10:1cc5:4ab9:fee9:5398) has joined #ceph
[5:10] * diegows (~diegows@190.190.2.126) Quit (Ping timeout: 480 seconds)
[5:12] * jlogan (~Thunderbi@2600:c00:3010:1:1::40) has joined #ceph
[5:13] * Vanony_ (~vovo@88.130.220.190) Quit (Ping timeout: 480 seconds)
[5:41] <mech422> Did anyone happen to make .debs for wheezy with rdb support for qemu and libvirt ?
[5:53] * Q310 (~Qten@ip-121-0-1-110.static.dsl.onqcomms.net) has joined #ceph
[5:56] * Qten (~Qten@ip-121-0-1-110.static.dsl.onqcomms.net) Quit (Read error: Connection reset by peer)
[5:57] * jks (~jks@3e6b5724.rev.stofanet.dk) Quit (Read error: Connection reset by peer)
[5:57] * jks (~jks@3e6b5724.rev.stofanet.dk) has joined #ceph
[5:58] * john_barbee (~jbarbee@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Read error: Connection reset by peer)
[5:58] * john_barbee (~jbarbee@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[5:59] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) has joined #ceph
[6:02] * tdb (~tdb@willow.kent.ac.uk) Quit (Remote host closed the connection)
[6:05] * jamespag` (~jamespage@culvain.gromper.net) Quit (Quit: Coyote finally caught me)
[6:08] * tdb (~tdb@willow.kent.ac.uk) has joined #ceph
[6:10] * jjgalvez1 (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) has joined #ceph
[6:11] * renzhi (~renzhi@116.226.35.53) has joined #ceph
[6:11] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[6:14] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Quit: noahmehl)
[6:16] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[6:25] * terje-_ (~terje@63-154-144-138.mpls.qwest.net) has joined #ceph
[6:26] * terje_ (~joey@63-154-144-138.mpls.qwest.net) has joined #ceph
[6:34] * terje-_ (~terje@63-154-144-138.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[6:35] * terje_ (~joey@63-154-144-138.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[6:36] * terje-_ (~terje@63-154-144-138.mpls.qwest.net) has joined #ceph
[6:37] * terje_ (~joey@63-154-144-138.mpls.qwest.net) has joined #ceph
[6:37] * loicd (~loic@2a01:e35:2eba:db10:1cc5:4ab9:fee9:5398) Quit (Quit: Leaving.)
[6:37] * loicd (~loic@magenta.dachary.org) has joined #ceph
[6:40] * terje-_ (~terje@63-154-144-138.mpls.qwest.net) Quit (Read error: Operation timed out)
[6:45] * terje_ (~joey@63-154-144-138.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[6:46] * loicd finally understood https://github.com/dachary/ceph/blob/b1f12a8cc9f544e86666706c97543bae26085fab/src/osd/PGLog.cc#L279 :-)
[6:46] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[6:59] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[7:01] * mech422 (~guest@65.19.151.114) Quit (Remote host closed the connection)
[7:06] * terje-_ (~terje@63-154-144-138.mpls.qwest.net) has joined #ceph
[7:14] * terje-_ (~terje@63-154-144-138.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[7:16] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) Quit (Quit: bia)
[7:17] * terje_ (~joey@63-154-144-138.mpls.qwest.net) has joined #ceph
[7:25] * dosaboy (~dosaboy@host86-161-201-199.range86-161.btcentralplus.com) Quit (Quit: leaving)
[7:25] * terje_ (~joey@63-154-144-138.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[7:37] * Cube (~Cube@cpe-76-95-217-129.socal.res.rr.com) has joined #ceph
[7:38] * terje_ (~joey@63-154-148-204.mpls.qwest.net) has joined #ceph
[7:46] * Cube (~Cube@cpe-76-95-217-129.socal.res.rr.com) Quit (Quit: Leaving.)
[7:46] * terje_ (~joey@63-154-148-204.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[7:59] * davidzlap (~Adium@ip68-96-75-123.oc.oc.cox.net) Quit (Quit: Leaving.)
[8:53] * terje_ (~joey@63-154-148-204.mpls.qwest.net) has joined #ceph
[9:01] * terje_ (~joey@63-154-148-204.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[9:14] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) has joined #ceph
[9:17] * jlogan (~Thunderbi@2600:c00:3010:1:1::40) Quit (Ping timeout: 480 seconds)
[9:36] * terje-_ (~terje@63-154-150-2.mpls.qwest.net) has joined #ceph
[9:41] * terje-_ (~terje@63-154-150-2.mpls.qwest.net) Quit (Read error: Operation timed out)
[9:50] * renzhi (~renzhi@116.226.35.53) Quit (Quit: Leaving)
[9:51] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[9:55] * tnt (~tnt@91.176.24.98) has joined #ceph
[10:05] * LeaChim (~LeaChim@176.250.167.111) has joined #ceph
[10:05] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[10:05] * loicd (~loic@magenta.dachary.org) has joined #ceph
[10:23] * julian_ (~julian@125.70.134.203) Quit (Quit: Leaving)
[10:32] * ScOut3R (~ScOut3R@gprsc2b0e2d3.pool.t-umts.hu) has joined #ceph
[10:38] * ScOut3R (~ScOut3R@gprsc2b0e2d3.pool.t-umts.hu) Quit (Remote host closed the connection)
[10:38] * terje_ (~joey@63-154-150-2.mpls.qwest.net) has joined #ceph
[10:46] * terje-_ (~terje@63-154-150-2.mpls.qwest.net) has joined #ceph
[10:47] * terje_ (~joey@63-154-150-2.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[10:54] * terje-_ (~terje@63-154-150-2.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[11:07] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) has joined #ceph
[11:10] * dcasier (~dcasier@ADijon-653-1-18-15.w86-213.abo.wanadoo.fr) has joined #ceph
[11:27] * terje-_ (~terje@63-154-150-2.mpls.qwest.net) has joined #ceph
[11:27] * Kioob (~kioob@2a01:e35:2432:58a0:21e:8cff:fe07:45b6) Quit (Read error: Connection reset by peer)
[11:35] * terje-_ (~terje@63-154-150-2.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[11:46] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[12:08] * LeaChim (~LeaChim@176.250.167.111) Quit (Ping timeout: 480 seconds)
[12:22] * terje- (~terje@63-154-152-90.mpls.qwest.net) has joined #ceph
[12:28] * LeaChim (~LeaChim@176.250.167.111) has joined #ceph
[12:30] * terje- (~terje@63-154-152-90.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[12:49] * terje_ (~joey@63-154-152-90.mpls.qwest.net) has joined #ceph
[12:52] * terje- (~terje@63-154-152-90.mpls.qwest.net) has joined #ceph
[12:56] * terje_ (~joey@63-154-152-90.mpls.qwest.net) Quit (Read error: Operation timed out)
[13:00] * terje- (~terje@63-154-152-90.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[13:44] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Remote host closed the connection)
[13:48] * jahkeup (~jahkeup@209.248.26.24) has joined #ceph
[13:50] * miniyo (~miniyo@0001b53b.user.oftc.net) Quit (Quit: WeeChat 0.4.0)
[13:59] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[14:03] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Ping timeout: 480 seconds)
[14:13] * terje- (~terje@63-154-133-180.mpls.qwest.net) has joined #ceph
[14:21] * terje- (~terje@63-154-133-180.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[14:38] * Maskul (~Maskul@host-89-241-165-79.as13285.net) has joined #ceph
[14:41] * dcasier (~dcasier@ADijon-653-1-18-15.w86-213.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[14:48] * terje-_ (~terje@75-166-102-61.hlrn.qwest.net) has joined #ceph
[14:49] * terje_ (~joey@75-166-102-61.hlrn.qwest.net) has joined #ceph
[14:51] * jluis (~JL@89.181.150.251) has joined #ceph
[14:56] * joao (~JL@89.181.151.177) Quit (Ping timeout: 480 seconds)
[15:02] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[15:02] * loicd (~loic@magenta.dachary.org) has joined #ceph
[15:03] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[15:13] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[15:36] * john_barbee_ (~jbarbee@c-98-226-73-253.hsd1.in.comcast.net) has joined #ceph
[15:38] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[15:39] * julian (~julianwa@125.70.134.203) has joined #ceph
[15:48] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[15:51] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[15:52] * loicd (~loic@magenta.dachary.org) has joined #ceph
[16:08] * Guest1449 (mark@tilia.nedworks.org) Quit (Quit: leaving)
[16:10] * dcasier (~dcasier@ADijon-653-1-18-15.w86-213.abo.wanadoo.fr) has joined #ceph
[16:24] * mschiff (~mschiff@port-1469.pppoe.wtnet.de) has joined #ceph
[16:25] * julian (~julianwa@125.70.134.203) Quit (Quit: afk)
[16:28] * mschiff_ (~mschiff@port-1469.pppoe.wtnet.de) Quit (Ping timeout: 480 seconds)
[16:35] * mschiff_ (~mschiff@port-1469.pppoe.wtnet.de) has joined #ceph
[16:35] * mschiff (~mschiff@port-1469.pppoe.wtnet.de) Quit (Read error: Connection reset by peer)
[16:37] * diegows (~diegows@190.190.2.126) has joined #ceph
[16:47] * jjgalvez1 (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Quit: Leaving.)
[16:47] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) has joined #ceph
[16:55] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[17:14] * The_Bishop (~bishop@f052103195.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[17:21] * DarkAce-Z (~BillyMays@50.107.54.92) Quit (Ping timeout: 480 seconds)
[17:33] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[17:44] * john_barbee (~jbarbee@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Quit: ChatZilla 0.9.90 [Firefox 21.0/20130511120803])
[17:51] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[17:54] * noahmehl_ (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[17:57] * yehuda_hm (~yehuda@2602:306:330b:1410:5183:e6bc:8046:69b) Quit (Read error: Connection timed out)
[18:00] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Ping timeout: 480 seconds)
[18:00] * noahmehl_ is now known as noahmehl
[18:03] * Vjarjadian (~IceChat77@90.214.208.5) Quit (Quit: Always try to be modest, and be proud about it!)
[18:08] * BillK (~BillK@124-148-124-185.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[18:10] * mschiff_ (~mschiff@port-1469.pppoe.wtnet.de) Quit (Remote host closed the connection)
[18:15] * mschiff (~mschiff@port-1469.pppoe.wtnet.de) has joined #ceph
[18:16] * yehuda_hm (~yehuda@2602:306:330b:1410:818f:80a6:8f91:d506) has joined #ceph
[18:17] * The_Bishop (~bishop@e177089176.adsl.alicedsl.de) has joined #ceph
[18:21] * mschiff (~mschiff@port-1469.pppoe.wtnet.de) Quit (Remote host closed the connection)
[18:26] * jahkeup (~jahkeup@209.248.26.24) Quit (Ping timeout: 480 seconds)
[18:31] * The_Bishop (~bishop@e177089176.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[18:36] * yehuda_hm (~yehuda@2602:306:330b:1410:818f:80a6:8f91:d506) Quit (Ping timeout: 480 seconds)
[18:50] * The_Bishop (~bishop@e177089176.adsl.alicedsl.de) has joined #ceph
[18:50] * yehuda_hm (~yehuda@2602:306:330b:1410:818f:80a6:8f91:d506) has joined #ceph
[19:07] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[19:19] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) has joined #ceph
[19:19] * ChanServ sets mode +v leseb
[19:25] * loicd (~loic@magenta.dachary.org) has joined #ceph
[19:36] * The_Bishop (~bishop@e177089176.adsl.alicedsl.de) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[19:42] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) has joined #ceph
[19:46] * jamespage (~jamespage@culvain.gromper.net) has joined #ceph
[19:52] * fridudad (~oftc-webi@p5B09D824.dip0.t-ipconnect.de) has joined #ceph
[19:52] <fridudad> sage sagewk FYI your branch wip-5176-cuttlefish does not build - wanted to test
[20:05] <mrjack_> yeah
[20:05] <mrjack_> i noticed that right now too
[20:08] <mrjack_> but i think that got merged to cuttlefish
[20:08] <mrjack_> oh, to next
[20:09] * dcasier (~dcasier@ADijon-653-1-18-15.w86-213.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[20:19] * dcasier (~dcasier@ADijon-653-1-18-15.w86-213.abo.wanadoo.fr) has joined #ceph
[20:30] * sjusthm (~sam@71-83-191-116.dhcp.gldl.ca.charter.com) has joined #ceph
[20:31] * dosaboy (~dosaboy@12.15.145.130) has joined #ceph
[20:51] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[20:54] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[20:57] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[21:19] <fridudad> mrjack_ yes but he wanted to try the patches with cuttlefish as well over the weekend
[21:19] * DarkAceZ (~BillyMays@50.107.53.195) has joined #ceph
[21:23] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[21:33] <fridudad> sjust sjusthm ping
[21:34] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) has joined #ceph
[21:35] * loicd (~loic@2a01:e35:2eba:db10:1cc5:4ab9:fee9:5398) has joined #ceph
[21:36] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[21:40] * loicd (~loic@2a01:e35:2eba:db10:1cc5:4ab9:fee9:5398) Quit (Quit: Leaving.)
[21:52] * redeemed (~redeemed@cpe-192-136-224-78.tx.res.rr.com) Quit (Quit: bia)
[21:57] * andreask (~andreask@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[21:57] * ChanServ sets mode +v andreask
[21:58] * andreask (~andreask@h081217068225.dyn.cm.kabsi.at) has left #ceph
[22:05] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[22:08] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit ()
[22:18] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[22:21] <fridudad> sage sagewk FYI your branch wip-5176-cuttlefish does not build - wanted to test
[22:33] <fridudad> osd recovering seems to be extremely CPU intensive since cuttlefish
[22:33] * tnt noticed that too
[22:33] <fridudad> so intensive that I/O gets blocked as it can't be served by the osd which is up again but has stuff to recover
[22:33] <fridudad> that seem to be the reason i reported stalled I/O yesterday
[22:34] <fridudad> the osd thread uses 200% CPU permantly until all pgs are recovered on a 3,6Ghz Intel Xeon
[22:44] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) Quit (Quit: Leaving.)
[22:51] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[22:53] * loicd (~loic@magenta.dachary.org) has joined #ceph
[22:58] <fridudad> tnt did you speak about that with anybody at ceph?
[22:59] <tnt> well, here in the channel, but nothing more.
[23:00] <fridudad> tnt OK
[23:03] * dcasier (~dcasier@ADijon-653-1-18-15.w86-213.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[23:11] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[23:12] * athrift (~nz_monkey@222.47.255.123.static.snap.net.nz) Quit (Remote host closed the connection)
[23:17] * terje- (~root@135.109.216.239) has joined #ceph
[23:17] <terje-> hi, I have a system stuck in clientreplay mode.
[23:18] <terje-> it will work for a minute or so after a restart and then it'll get marked laggy/crashed.
[23:18] <terje-> Not sure how to troubleshoot it
[23:25] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) has joined #ceph
[23:37] * eschnou (~eschnou@168.176-201-80.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[23:39] * fridudad (~oftc-webi@p5B09D824.dip0.t-ipconnect.de) Quit (Quit: Page closed)
[23:39] * fridudad (~oftc-webi@p5B09D824.dip0.t-ipconnect.de) has joined #ceph
[23:49] * fridudad (~oftc-webi@p5B09D824.dip0.t-ipconnect.de) Quit (Remote host closed the connection)
[23:56] * eegiks (~quassel@2a01:e35:8a2c:b230:499:a2c0:7e4d:7601) Quit (Ping timeout: 480 seconds)
[23:56] * eegiks (~quassel@2a01:e35:8a2c:b230:566:484c:c010:7ca6) has joined #ceph
[23:59] * BillK (~BillK@124-148-124-185.dyn.iinet.net.au) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.