#ceph IRC Log

Index

IRC Log for 2013-03-14

Timestamps are in GMT/BST.

[0:00] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[0:00] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) has joined #ceph
[0:04] * jantje_ (~jan@paranoid.nl) has joined #ceph
[0:08] * danieagle (~Daniel@177.99.133.222) Quit (Quit: Inte+ :-) e Muito Obrigado Por Tudo!!! ^^)
[0:08] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) Quit (Ping timeout: 480 seconds)
[0:09] <mjblw> it seems like reducing the max_backfill to 1 eventually calmed things enough so client IO is no longer starved
[0:09] <mjblw> i think it took a little while for all the backed up io to flush
[0:10] <mjblw> i guess it's probably good practice to reduce the max_backfill if you IN a bunch of OSDs at once
[0:10] * jantje (~jan@paranoid.nl) Quit (Ping timeout: 480 seconds)
[0:11] <gregaf> it's a tunable that you need to set up so it matches your hardware, yes
[0:11] <mjblw> what hardware does the default settings match? I have a bunch of enterprise class 7200 rpm spinning disk.
[0:12] <mjblw> these things push over 400 iops and almost 200 MB/s
[0:12] * jlogan (~Thunderbi@72.5.59.176) Quit (Ping timeout: 480 seconds)
[0:12] * Cube (~Cube@12.248.40.138) Quit (Quit: Leaving.)
[0:12] <mjblw> journals are on ssds
[0:12] * Cube (~Cube@12.248.40.138) has joined #ceph
[0:12] * Cube (~Cube@12.248.40.138) Quit ()
[0:13] <iggy> it's a tunable with a bad default?
[0:13] <mjblw> seems like i have pretty decent hardware. i'm wondering about the default, and whether it's bad...
[0:13] <iggy> yours is (I believe) the generally accepted setup
[0:13] <gregaf> it might be bad
[0:14] <gregaf> or it might be that they interact in complex ways with other stuff, so this works well for 1Gig but bad on 10Gig (for instance) because 1Gig provides a different throttle
[0:15] <iggy> hmm, yeah, I think most people might still be on 1G/bonded setups
[0:18] * jlogan (~Thunderbi@72.5.59.176) has joined #ceph
[0:20] <sstan_> ^ that's what I'm experimenting with
[0:23] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) has joined #ceph
[0:33] <mjevans> Anyone have a reccommendation besides libvirt for managing qemu instances and allowing for 'easy' migration with rbd storage? -- After trying to get dependencies to compile (a bunch of Haskell stuff that is halfway between converting from 'haskell' to 'ghc' it seems) I think I'm giving up on ganeti in addition to libvirt... I briefly considered OpenStack, but as it seems to depend on libvirt I decided against that.
[0:34] <mjevans> For the moment I'm seriously considering just 'rolling my own' shell scripts to abstract out the syntax and automate the very basics of using Qemu
[0:34] <mjevans> libvirt... well it complained about various locking issues in a way that made me feel it didn't understand that a 'network' type disk should never be migrated.
[0:35] * mcclurmc_laptop (~mcclurmc@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) Quit (Read error: Operation timed out)
[0:53] * Cube (~Cube@66-87-66-23.pools.spcsdns.net) has joined #ceph
[0:56] * frank9999 (~frank@kantoor.transip.nl) Quit (Read error: Connection reset by peer)
[0:59] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) Quit (Remote host closed the connection)
[1:00] <iggy> there's a list of mgmt layers on the linux-kvm.org wiki
[1:01] <iggy> libvirt is the only one I know of that "supports" qemu's built-in librbd functionality
[1:01] <iggy> others would probably work with kernel rbd
[1:02] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[1:21] * dmick (~dmick@2607:f298:a:607:a40d:8efc:5e0f:154e) Quit (Quit: Leaving.)
[1:24] * jjgalvez (~jjgalvez@12.248.40.138) Quit (Quit: Leaving.)
[1:30] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) has joined #ceph
[1:36] <sstan_> a shell script should do the trick
[1:38] * livekcats (~stackevil@178.112.19.126.wireless.dyn.drei.com) has joined #ceph
[1:40] * The_Bishop (~bishop@2001:470:50b6:0:b420:1010:bbab:da7b) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[1:41] * alram (~alram@38.122.20.226) Quit (Quit: leaving)
[1:43] * stackevil (~stackevil@77.116.2.173.wireless.dyn.drei.com) Quit (Ping timeout: 480 seconds)
[1:47] * xiaoxi (~xiaoxiche@134.134.137.73) has joined #ceph
[1:48] * noob2 (~cjh@173.252.71.4) Quit (Quit: Leaving.)
[1:49] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) has joined #ceph
[1:52] * rturk is now known as rturk-away
[2:01] * nwat (~Adium@eduroam-242-170.ucsc.edu) Quit (Quit: Leaving.)
[2:03] * mgalkiewicz (~mgalkiewi@toya.hederanetworks.net) Quit (Ping timeout: 480 seconds)
[2:08] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[2:18] * noahmehl_ (~noahmehl@cpe-75-186-45-161.cinci.res.rr.com) has joined #ceph
[2:18] * Cube (~Cube@66-87-66-23.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[2:18] * Cube (~Cube@66-87-65-77.pools.spcsdns.net) has joined #ceph
[2:19] <xiaoxi> hi, I have upgrade my ceph to 0.58 and usually seem some osd daemon died for "sync_entry timed out after 600 seconds"
[2:19] * Cube (~Cube@66-87-65-77.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[2:19] <xiaoxi> is there a bug relate to it?
[2:22] * elder (~elder@c-71-195-31-37.hsd1.mn.comcast.net) Quit (Quit: Leaving)
[2:22] * jlogan1 (~Thunderbi@72.5.59.176) has joined #ceph
[2:22] * noahmehl (~noahmehl@cpe-75-186-45-161.cinci.res.rr.com) Quit (Ping timeout: 480 seconds)
[2:22] * noahmehl_ is now known as noahmehl
[2:22] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) Quit (Remote host closed the connection)
[2:24] * jlogan (~Thunderbi@72.5.59.176) Quit (Ping timeout: 480 seconds)
[2:25] * elder (~elder@c-71-195-31-37.hsd1.mn.comcast.net) has joined #ceph
[2:25] * ChanServ sets mode +o elder
[2:33] * nwat (~Adium@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[2:33] * nwat (~Adium@c-50-131-197-174.hsd1.ca.comcast.net) has left #ceph
[2:36] * TiMOiD (tim@2403-9000-1000-0000-181e-940d-3e68-342e.ipv6.onqnetworks.net) has joined #ceph
[2:41] * Cube (~Cube@66-87-64-10.pools.spcsdns.net) has joined #ceph
[2:45] * Cube (~Cube@66-87-64-10.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[2:49] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[2:51] * KevinPerks1 (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[2:53] * LeaChim (~LeaChim@b01bdcc0.bb.sky.com) Quit (Ping timeout: 480 seconds)
[3:02] * dpippenger (~riven@cpe-76-166-221-185.socal.res.rr.com) Quit (Remote host closed the connection)
[3:06] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[3:13] * The_Bishop (~bishop@e179003093.adsl.alicedsl.de) has joined #ceph
[3:18] * diegows (~diegows@190.190.2.126) Quit (Ping timeout: 480 seconds)
[3:20] * JohansGlock_ (~quassel@kantoor.transip.nl) Quit (Read error: Connection reset by peer)
[3:21] * chutzpah (~chutz@199.21.234.7) Quit (Quit: Leaving)
[3:26] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[3:28] * Philip_ (~Philip@hnvr-4dbd07c1.pool.mediaWays.net) Quit (Ping timeout: 480 seconds)
[3:28] * tore_ (~tore@110.50.71.129) has joined #ceph
[3:35] * wschulze (~wschulze@cpe-69-203-80-81.nyc.res.rr.com) Quit (Quit: Leaving.)
[3:43] <mjevans> xiaoxi: have you checked your clocks are staying synced (or are synced to begin with), as well as possibly monitoring the performance of it while it's joined?
[4:00] * xanpeng (~xanpeng@a.clients.kiwiirc.com) has joined #ceph
[4:05] * xanpeng (~xanpeng@a.clients.kiwiirc.com) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[4:07] * xan (~xan@a.clients.kiwiirc.com) has joined #ceph
[4:11] * miroslav (~miroslav@173-228-38-131.dsl.dynamic.sonic.net) has joined #ceph
[4:12] * xan (~xan@a.clients.kiwiirc.com) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[4:14] * sjustlaptop (~sam@71-83-191-116.dhcp.gldl.ca.charter.com) Quit (Quit: Leaving.)
[4:26] * yanzheng (~zhyan@134.134.137.71) has joined #ceph
[4:29] * jlogan1 (~Thunderbi@72.5.59.176) Quit (Ping timeout: 480 seconds)
[4:40] * jlogan (~Thunderbi@72.5.59.176) has joined #ceph
[4:52] * miroslav (~miroslav@173-228-38-131.dsl.dynamic.sonic.net) Quit (Quit: Leaving.)
[4:59] * slang1 (~slang@207-229-177-80.c3-0.drb-ubr1.chi-drb.il.cable.rcn.com) Quit (Ping timeout: 480 seconds)
[5:02] * ramonskiee (3e295201@ircip2.mibbit.com) Quit (Quit: http://www.mibbit.com ajax IRC Client)
[5:05] * wschulze (~wschulze@cpe-69-203-80-81.nyc.res.rr.com) has joined #ceph
[5:06] * wschulze (~wschulze@cpe-69-203-80-81.nyc.res.rr.com) Quit ()
[5:28] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[5:58] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[6:03] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) has joined #ceph
[6:10] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[6:14] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) has joined #ceph
[6:15] * sleinen1 (~Adium@2001:620:0:25:a531:2330:3387:a07a) has joined #ceph
[6:16] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[6:16] * nick5 (~nick@74.222.153.12) has joined #ceph
[6:21] * sleinen2 (~Adium@user-28-10.vpn.switch.ch) has joined #ceph
[6:22] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[6:25] * sleinen1 (~Adium@2001:620:0:25:a531:2330:3387:a07a) Quit (Ping timeout: 480 seconds)
[6:46] * jlogan (~Thunderbi@72.5.59.176) Quit (Ping timeout: 480 seconds)
[6:47] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[6:47] * nick5 (~nick@74.222.153.12) has joined #ceph
[6:48] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[6:48] * nick5 (~nick@74.222.153.12) has joined #ceph
[6:48] * jlogan (~Thunderbi@72.5.59.176) has joined #ceph
[7:09] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[7:10] * nick5 (~nick@74.222.153.12) has joined #ceph
[7:11] * KindTwo (KindOne@h246.2.40.162.dynamic.ip.windstream.net) has joined #ceph
[7:17] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[7:17] * KindTwo is now known as KindOne
[7:17] * loicd (~loic@AAnnecy-257-1-112-254.w90-36.abo.wanadoo.fr) has joined #ceph
[7:18] * sleinen (~Adium@2001:620:0:26:fcfa:1a3e:d25:e344) has joined #ceph
[7:22] * sleinen2 (~Adium@user-28-10.vpn.switch.ch) Quit (Read error: Operation timed out)
[7:25] * yanzheng (~zhyan@134.134.137.71) Quit (Remote host closed the connection)
[7:44] * yanzheng (~zhyan@jfdmzpr03-ext.jf.intel.com) has joined #ceph
[7:47] * loicd (~loic@AAnnecy-257-1-112-254.w90-36.abo.wanadoo.fr) Quit (Quit: Leaving.)
[8:01] * themgt (~themgt@24-177-232-181.dhcp.gnvl.sc.charter.com) Quit (Quit: Pogoapp - http://www.pogoapp.com)
[8:03] * loicd (~loic@AAnnecy-257-1-112-254.w90-36.abo.wanadoo.fr) has joined #ceph
[8:04] * raso (~raso@deb-multimedia.org) Quit (Quit: WeeChat 0.3.8)
[8:05] * raso (~raso@deb-multimedia.org) has joined #ceph
[8:06] * sleinen (~Adium@2001:620:0:26:fcfa:1a3e:d25:e344) Quit (Quit: Leaving.)
[8:24] * xiaoxi (~xiaoxiche@134.134.137.73) Quit (Ping timeout: 480 seconds)
[8:26] * sleinen (~Adium@130.59.94.87) has joined #ceph
[8:27] * sleinen1 (~Adium@2001:620:0:26:ad3c:85aa:a518:f3c7) has joined #ceph
[8:28] * yanzheng (~zhyan@jfdmzpr03-ext.jf.intel.com) Quit (Remote host closed the connection)
[8:34] * sleinen (~Adium@130.59.94.87) Quit (Ping timeout: 480 seconds)
[8:42] * yanzheng (~zhyan@134.134.137.71) has joined #ceph
[8:43] * xiaoxi (~xiaoxiche@134.134.137.73) has joined #ceph
[8:43] * livekcats (~stackevil@178.112.19.126.wireless.dyn.drei.com) Quit (Quit: This Mac has gone to sleep!)
[8:46] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Quit: Leaving.)
[8:46] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) has joined #ceph
[8:49] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) has joined #ceph
[8:50] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) Quit (Remote host closed the connection)
[8:54] * jlogan (~Thunderbi@72.5.59.176) Quit (Read error: Connection reset by peer)
[8:54] * jlogan (~Thunderbi@72.5.59.176) has joined #ceph
[8:56] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[8:59] * sleinen1 (~Adium@2001:620:0:26:ad3c:85aa:a518:f3c7) Quit (Quit: Leaving.)
[9:01] * sleinen (~Adium@130.59.94.87) has joined #ceph
[9:02] * sleinen1 (~Adium@2001:620:0:26:99ce:ecd0:b701:52aa) has joined #ceph
[9:02] * sleinen1 (~Adium@2001:620:0:26:99ce:ecd0:b701:52aa) Quit ()
[9:02] * sleinen1 (~Adium@130.59.94.87) has joined #ceph
[9:02] * sleinen (~Adium@130.59.94.87) Quit (Read error: Connection reset by peer)
[9:03] * sleinen (~Adium@130.59.94.87) has joined #ceph
[9:03] * sleinen1 (~Adium@130.59.94.87) Quit (Read error: Connection reset by peer)
[9:03] * sleinen (~Adium@130.59.94.87) Quit ()
[9:03] * sleinen (~Adium@130.59.94.87) has joined #ceph
[9:06] * Vjarjadian (~IceChat77@5ad6d005.bb.sky.com) Quit (Quit: On the other hand, you have different fingers.)
[9:11] * sleinen (~Adium@130.59.94.87) Quit (Ping timeout: 480 seconds)
[9:13] * loicd (~loic@AAnnecy-257-1-112-254.w90-36.abo.wanadoo.fr) Quit (Quit: Leaving.)
[9:16] * ramonskiee (3e295201@ircip2.mibbit.com) has joined #ceph
[9:16] * ramonskiee (3e295201@ircip2.mibbit.com) Quit ()
[9:17] * ramonskiee (3e295201@ircip2.mibbit.com) has joined #ceph
[9:17] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[9:21] * gerard_dethier (~Thunderbi@85.234.217.115.static.edpnet.net) has joined #ceph
[9:24] * jjgalvez1 (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) has joined #ceph
[9:25] * ramonskiee (3e295201@ircip2.mibbit.com) Quit (Quit: http://www.mibbit.com ajax IRC Client)
[9:26] * jbd_ (~jbd_@34322hpv162162.ikoula.com) has joined #ceph
[9:27] * esammy (~esamuels@host-2-102-70-24.as13285.net) has left #ceph
[9:29] * leseb (~leseb@3.46-14-84.ripe.coltfrance.com) has joined #ceph
[9:29] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[9:33] * mgalkiewicz (~mgalkiewi@toya.hederanetworks.net) has joined #ceph
[9:33] * l0nk (~alex@83.167.43.235) has joined #ceph
[9:35] * leseb_ (~leseb@3.46-14-84.ripe.coltfrance.com) has joined #ceph
[9:36] * hybrid5121 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[9:40] * frank9999 (~frank@kantoor.transip.nl) has joined #ceph
[9:42] * leseb (~leseb@3.46-14-84.ripe.coltfrance.com) Quit (Ping timeout: 480 seconds)
[9:45] * gucki (~smuxi@HSI-KBW-078-042-029-115.hsi3.kabel-badenwuerttemberg.de) has joined #ceph
[9:58] * hybrid5121 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[9:59] * hybrid5121 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[10:02] * yanzheng (~zhyan@134.134.137.71) Quit (Quit: Leaving)
[10:04] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) has joined #ceph
[10:05] * xiaoxi (~xiaoxiche@134.134.137.73) Quit (Remote host closed the connection)
[10:07] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) Quit (Quit: tryggvil)
[10:07] * mcclurmc_laptop (~mcclurmc@firewall.ctxuk.citrix.com) has joined #ceph
[10:09] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) has joined #ceph
[10:10] * jjgalvez1 (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[10:13] * Rocky (~r.nap@188.205.52.204) Quit (Quit: whoopsie)
[10:13] * Rocky (~r.nap@188.205.52.204) has joined #ceph
[10:14] * dosaboy (~gizmo@faun.canonical.com) has joined #ceph
[10:16] * oddover (~oddover@glados.colorado.edu) Quit (Quit: ~ Trillian - www.trillian.im ~)
[10:18] * jjgalvez (~jjgalvez@cpe-76-175-30-67.socal.res.rr.com) Quit (Read error: Connection reset by peer)
[10:27] * LeaChim (~LeaChim@b01bdcc0.bb.sky.com) has joined #ceph
[10:30] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[10:32] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[10:40] * mgalkiewicz (~mgalkiewi@toya.hederanetworks.net) Quit (Remote host closed the connection)
[10:55] * ramonskiee (3e295201@ircip4.mibbit.com) has joined #ceph
[10:57] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[11:00] * jlogan (~Thunderbi@72.5.59.176) Quit (Read error: Connection reset by peer)
[11:00] * jlogan (~Thunderbi@72.5.59.176) has joined #ceph
[11:12] * ramonskiee (3e295201@ircip4.mibbit.com) Quit (Quit: http://www.mibbit.com ajax IRC Client)
[11:14] * diegows (~diegows@190.190.2.126) has joined #ceph
[11:14] * roshan (~root@14.140.216.194) has joined #ceph
[11:21] * jlogan (~Thunderbi@72.5.59.176) Quit (Quit: jlogan)
[11:23] * stackevil (~stackevil@cpe90-146-43-165.liwest.at) has joined #ceph
[11:23] <roshan> who
[11:23] <roshan> \who
[11:24] * sleinen (~Adium@2001:620:0:25:3121:38ad:8280:42e2) has joined #ceph
[11:24] <roshan> \/quit
[11:24] * roshan (~root@14.140.216.194) Quit (Quit: leaving)
[11:25] * root__ (~root@14.140.216.194) has joined #ceph
[11:25] * root__ is now known as rtm
[11:26] * rtm (~root@14.140.216.194) Quit ()
[11:27] * rtm (~rtm@14.140.216.194) has joined #ceph
[11:30] * noahmehl (~noahmehl@cpe-75-186-45-161.cinci.res.rr.com) Quit (Quit: noahmehl)
[11:32] * noahmehl (~noahmehl@cpe-75-186-45-161.cinci.res.rr.com) has joined #ceph
[11:41] * gaveen (~gaveen@182.161.25.135) has joined #ceph
[11:56] * sleinen (~Adium@2001:620:0:25:3121:38ad:8280:42e2) Quit (Quit: Leaving.)
[11:59] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) Quit (Quit: tryggvil)
[12:00] * sleinen (~Adium@130.59.94.87) has joined #ceph
[12:03] * hybrid5121 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[12:05] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[12:07] * sleinen1 (~Adium@2001:620:0:25:b8d9:25:cfd2:2f87) has joined #ceph
[12:08] * rtm_ (~rtm@14.140.216.194) has joined #ceph
[12:08] * sleinen (~Adium@130.59.94.87) Quit (Ping timeout: 480 seconds)
[12:09] * rtm (~rtm@14.140.216.194) Quit (Ping timeout: 480 seconds)
[12:10] <Elbandi_> i got a question about mds replay config
[12:10] <Elbandi_> [mds.alpha] host = alpha
[12:10] <Elbandi_> [mds.charlie] host = charlie mds standby replay = true mds standby for name = alpha
[12:11] <Elbandi_> this means, charlie is a "hot spare" for alpha?
[12:11] * leseb_ (~leseb@3.46-14-84.ripe.coltfrance.com) Quit (Remote host closed the connection)
[12:13] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[12:16] * rtm_ (~rtm@14.140.216.194) Quit (Ping timeout: 480 seconds)
[12:36] * leseb (~leseb@3.46-14-84.ripe.coltfrance.com) has joined #ceph
[12:39] <joao> Elbandi_, if I understand the docs correctly, that is right
[12:39] <joao> http://ceph.com/docs/master/cephfs/mds-config-ref/
[12:43] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[12:50] * ramonskie (ab15507e@ircip1.mibbit.com) has joined #ceph
[12:55] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[12:58] * aliguori (~anthony@cpe-70-112-157-87.austin.res.rr.com) has joined #ceph
[12:58] * loicd (~loic@AAnnecy-257-1-112-254.w90-36.abo.wanadoo.fr) has joined #ceph
[13:06] * wschulze (~wschulze@cpe-69-203-80-81.nyc.res.rr.com) has joined #ceph
[13:13] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) Quit (Quit: tryggvil)
[13:21] * The_Bishop_ (~bishop@e179004198.adsl.alicedsl.de) has joined #ceph
[13:27] * markbby (~Adium@168.94.245.2) has joined #ceph
[13:28] * The_Bishop (~bishop@e179003093.adsl.alicedsl.de) Quit (Read error: Connection reset by peer)
[13:34] * diegows (~diegows@190.190.2.126) Quit (Ping timeout: 480 seconds)
[13:35] * stackevil (~stackevil@cpe90-146-43-165.liwest.at) Quit (Ping timeout: 480 seconds)
[13:35] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) Quit (Quit: Leaving.)
[13:51] <ramonskie> scuttlemonkey: my problem is solved :)
[13:53] * slang1 (~slang@207-229-177-80.c3-0.drb-ubr1.chi-drb.il.cable.rcn.com) has joined #ceph
[13:56] * stackevil (~stackevil@cpe90-146-49-158.liwest.at) has joined #ceph
[14:03] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[14:19] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[14:20] * JohansGlock (~quassel@kantoor.transip.nl) has joined #ceph
[14:22] * xiaoxi (~xiaoxiche@jfdmzpr06-ext.jf.intel.com) has joined #ceph
[14:27] <scuttlemonkey> ramonskie: awesome! What was it?
[14:32] * diegows (~diegows@190.190.2.126) has joined #ceph
[14:37] <ramonskie> well i have done now a clean install and followed the 5 minutes guide and then followed the ceph openstack guide and it worked
[14:37] <scuttlemonkey> ahh
[14:37] <ramonskie> yesterday i installed it with the chef script
[14:38] <scuttlemonkey> hmm
[14:38] <scuttlemonkey> didn't you install argo originally and upgrade?
[14:38] <ramonskie> yup
[14:38] <scuttlemonkey> I was wondering last night if maybe one of the osds was still stuck on argo
[14:38] <ramonskie> i installed argo with chef
[14:38] <scuttlemonkey> that would cause what was going on
[14:38] <ramonskie> hmm okay
[14:39] <ramonskie> well i wanted to install bobtail with the chef cookbook but they are broken
[14:39] <scuttlemonkey> really?
[14:39] <scuttlemonkey> I'll have to look into that
[14:39] <ramonskie> i created this issue http://tracker.ceph.com/issues/4420
[14:39] <scuttlemonkey> last I heard they were getting a lot of love
[14:40] <ramonskie> maby its just a strange combination of my crowbar/chef instance
[14:40] <scuttlemonkey> might be worth adding a comment saying what your env is
[14:42] <ramonskie> just updated it..
[14:43] <ramonskie> but how can i check if osd is still on argo?
[14:43] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[14:43] <scuttlemonkey> you can actually use 'ceph-osd -v'
[14:43] <scuttlemonkey> since it calls a different package
[14:44] <ramonskie> okay when i have some spare time i will try to install it again with ceph
[14:44] <ramonskie> chef
[14:45] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) Quit (Quit: Leaving.)
[14:45] <scuttlemonkey> cool
[14:45] <ramonskie> will let you know when i have some succes
[14:45] <scuttlemonkey> I know there are other folks running chef as their deployment tool....I'll have to see if anyone knows a good example
[14:46] <scuttlemonkey> excellent
[14:48] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) Quit (Quit: tryggvil)
[14:49] * fghaas (~florian@91-119-65-118.dynamic.xdsl-line.inode.at) has joined #ceph
[14:58] * drokita (~drokita@199.255.228.128) has joined #ceph
[15:00] * markbby (~Adium@168.94.245.2) Quit (Remote host closed the connection)
[15:00] * markbby (~Adium@168.94.245.2) has joined #ceph
[15:03] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[15:08] * The_Bishop_ (~bishop@e179004198.adsl.alicedsl.de) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[15:08] * xiaoxi (~xiaoxiche@jfdmzpr06-ext.jf.intel.com) Quit (Remote host closed the connection)
[15:13] * xiaoxi (~xiaoxiche@134.134.137.73) has joined #ceph
[15:13] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[15:28] * ramonskie (ab15507e@ircip1.mibbit.com) Quit (Quit: http://www.mibbit.com ajax IRC Client)
[15:28] * mikedawson_ (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[15:32] * drokita (~drokita@199.255.228.128) has left #ceph
[15:33] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[15:33] * mikedawson_ is now known as mikedawson
[15:34] <xiaoxi> for RBD,What will happen in OSD if 2 writes hit a same chunk? I remember someone told me that the second write must wait until the first finished,is it true?
[15:49] * BillK (~BillK@58-7-59-228.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[15:55] * sleinen1 (~Adium@2001:620:0:25:b8d9:25:cfd2:2f87) Quit (Quit: Leaving.)
[15:57] * BillK (~BillK@58-7-205-127.dyn.iinet.net.au) has joined #ceph
[16:05] * xiaoxi (~xiaoxiche@134.134.137.73) Quit (Ping timeout: 480 seconds)
[16:06] * sleinen (~Adium@130.59.94.87) has joined #ceph
[16:07] <scuttlemonkey> xiaoxi: yes, writes in that case get queued and will send an acknowledgement upon completion
[16:07] * sleinen1 (~Adium@2001:620:0:25:6589:6b89:2a15:fe4b) has joined #ceph
[16:08] <scuttlemonkey> this may help: http://ceph.com/docs/master/architecture/#how-ceph-scales
[16:12] * sleinen1 (~Adium@2001:620:0:25:6589:6b89:2a15:fe4b) Quit ()
[16:14] * sleinen (~Adium@130.59.94.87) Quit (Ping timeout: 480 seconds)
[16:16] * vata (~vata@2607:fad8:4:6:ce8:b0fc:4942:63ce) has joined #ceph
[16:21] * wer_ (~wer@wer.youfarted.net) Quit (Ping timeout: 480 seconds)
[16:26] * wer_ (~wer@wer.youfarted.net) has joined #ceph
[16:27] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[16:27] * nick5 (~nick@74.222.153.12) has joined #ceph
[16:28] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[16:29] * nick5 (~nick@74.222.153.12) has joined #ceph
[16:31] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[16:31] * nick5 (~nick@74.222.153.12) has joined #ceph
[16:34] * sleinen (~Adium@130.59.94.87) has joined #ceph
[16:36] * sleinen1 (~Adium@2001:620:0:26:603c:5874:2fbc:62e2) has joined #ceph
[16:41] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[16:42] * sleinen (~Adium@130.59.94.87) Quit (Ping timeout: 480 seconds)
[16:46] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[16:47] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[16:51] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit ()
[16:53] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[16:54] * gerard_dethier (~Thunderbi@85.234.217.115.static.edpnet.net) Quit (Quit: gerard_dethier)
[16:58] * serverascode (~serverasc@wwdyn182.cs.ualberta.ca) has joined #ceph
[16:59] * terje_ (~joey@184-96-148-241.hlrn.qwest.net) has joined #ceph
[17:05] * gucki (~smuxi@HSI-KBW-078-042-029-115.hsi3.kabel-badenwuerttemberg.de) Quit (Ping timeout: 480 seconds)
[17:05] * gucki (~smuxi@vpn.simplificator.com) has joined #ceph
[17:08] * BillK (~BillK@58-7-205-127.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[17:09] * aliguori (~anthony@cpe-70-112-157-87.austin.res.rr.com) Quit (Remote host closed the connection)
[17:21] * alram (~alram@38.122.20.226) has joined #ceph
[17:22] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) has joined #ceph
[17:28] * PerlStalker (~PerlStalk@72.166.192.70) has joined #ceph
[17:30] * wer_ (~wer@wer.youfarted.net) Quit (Read error: Operation timed out)
[17:31] * BillK (~BillK@58-7-239-136.dyn.iinet.net.au) has joined #ceph
[17:32] * aliguori (~anthony@32.97.110.51) has joined #ceph
[17:37] * noob2 (~cjh@173.252.71.3) has joined #ceph
[17:37] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[17:42] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[17:43] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[17:43] * wer (~wer@119.sub-70-192-211.myvzw.com) has joined #ceph
[17:46] <janos> any time i have to mess with httpd or znc's config files, i thank you guys for not being retarded like them
[17:46] <janos> that's a bigger compliment than it sounds like ;)
[17:47] * markbby (~Adium@168.94.245.2) Quit (Quit: Leaving.)
[17:47] * markbby (~Adium@168.94.245.2) has joined #ceph
[17:47] <mjevans> tmux + irssi or weechat
[17:48] <janos> hrmm, i may have to check out tmux
[17:51] <scuttlemonkey> haha
[17:51] <scuttlemonkey> thanks for the vote of confidence janos :)
[17:52] <janos> haha
[17:52] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) Quit (Remote host closed the connection)
[17:52] * mgalkiewicz (~mgalkiewi@staticline-31-182-128-35.toya.net.pl) has joined #ceph
[17:53] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) Quit (Quit: Leaving.)
[17:56] * sleinen1 (~Adium@2001:620:0:26:603c:5874:2fbc:62e2) Quit (Quit: Leaving.)
[17:56] * sleinen (~Adium@130.59.94.87) has joined #ceph
[18:01] * l0nk (~alex@83.167.43.235) Quit (Quit: Leaving.)
[18:04] * sleinen (~Adium@130.59.94.87) Quit (Ping timeout: 480 seconds)
[18:10] * jbd_ (~jbd_@34322hpv162162.ikoula.com) has left #ceph
[18:12] * dmick (~dmick@2607:f298:a:607:1dfc:aa5e:b2c7:b1bd) has joined #ceph
[18:13] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) has joined #ceph
[18:20] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) Quit (Quit: Leaving.)
[18:22] * The_Bishop (~bishop@2001:470:50b6:0:900f:3de0:4e76:c02f) has joined #ceph
[18:24] * BillK (~BillK@58-7-239-136.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[18:32] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) Quit (Remote host closed the connection)
[18:34] * Vjarjadian (~IceChat77@5ad6d005.bb.sky.com) has joined #ceph
[18:34] * loicd1 (~loic@AAnnecy-257-1-103-124.w90-36.abo.wanadoo.fr) has joined #ceph
[18:37] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) Quit (Quit: Leaving.)
[18:37] * chutzpah (~chutz@199.21.234.7) has joined #ceph
[18:38] * loicd (~loic@AAnnecy-257-1-112-254.w90-36.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[18:44] * barryo1 (~barry@host86-128-180-76.range86-128.btcentralplus.com) has joined #ceph
[18:45] * eschnou (~eschnou@148.91-201-80.adsl-dyn.isp.belgacom.be) has joined #ceph
[18:50] * mauilion (~dcooley@c-71-198-86-127.hsd1.ca.comcast.net) has joined #ceph
[18:51] <mauilion> Hey all, I changed some routing which meant that I had to bounce some of the cluster network interfaces. After that I see that I have some stuck pg's http://nopaste.linux-dev.org/?71103 I am reading this page to try to recover. http://ceph.com/docs/master/rados/operations/troubleshooting-osd/#failures-osd-unfound but I am still kind of stuck
[18:52] <barryo1> I think I've done something silly with my test cluster, i decided to see if I had captured everything in our configuration management system by rebuilding and restoring one of my test machines, I fogot that it was the only host running a mds, so have accidentally trashed all the data in /var/lib/ceph/mds/ceph-a. ceph health is reporting "HEALTH_WARN mds a is laggy; 1 mons down, quorum 0,1 b,d". I've not using cephfs at the mome
[18:55] <scuttlemonkey> mauilion: what happens when you 'ceph pg 1.858 query' ?
[18:55] * dpippenger (~riven@216.103.134.250) has joined #ceph
[18:55] <mauilion> 1 sec
[18:55] * eschnou (~eschnou@148.91-201-80.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[18:55] * leseb (~leseb@3.46-14-84.ripe.coltfrance.com) Quit (Remote host closed the connection)
[18:56] <mauilion> scuttlemonkey: a whole lot of nothing
[18:56] <mauilion> it's just hanging there
[18:56] <scuttlemonkey> hrm
[18:57] <scuttlemonkey> no output at all?
[18:57] <mauilion> not yet
[18:57] <dmick> did your mon(s) die?
[18:57] <scuttlemonkey> yeah, sounds like ceph can't get to the mons
[18:57] <mauilion> all three are "running"
[18:58] * Cube (~Cube@12.248.40.138) has joined #ceph
[18:58] * stackevil (~stackevil@cpe90-146-49-158.liwest.at) Quit (Quit: This Mac has gone to sleep!)
[18:58] * serverascode (~serverasc@wwdyn182.cs.ualberta.ca) Quit (Quit: Textual IRC Client: www.textualapp.com)
[18:58] <mauilion> scuttlemonkey: http://nopaste.linux-dev.org/?71104
[19:00] <scuttlemonkey> barryo1: so you have 1 mon down, and the mds got trashed before you brought it back up?
[19:00] <mauilion> sweet!
[19:00] <mauilion> I just got it
[19:00] <dmick> got it?
[19:00] <barryo1> yeah, i can fix the mon myself, it's the mds that I'm unsure of
[19:00] <mauilion> 2013-03-14 10:58:49.683162 osd.18 [WRN] slow request 1920.315668 seconds old, received at 2013-03-14 10:26:49.367416: osd_op(client.6415.1:132525 10000000402.0000074f [write 0~4194304 [1@-1]] 0.32f3bbb3 RETRY snapc 1=[]) currently reached pg
[19:01] <mauilion> I restarted osd.18
[19:01] <mauilion> and now I am back in good health
[19:01] <scuttlemonkey> good deal
[19:01] <dmick> ah. so it was probably trying to contact a moribund osd.18
[19:01] <dmick> to ask about the pg
[19:01] <scuttlemonkey> nod
[19:01] <mauilion> thanks all
[19:01] <dmick> as to why it was moribund, well, that's another issue, but evidence is gone now :)
[19:02] <mauilion> that's probably because I was monkeying with the underlying network
[19:03] <scuttlemonkey> barryo1: you haven't used cephfs yet though?
[19:03] * mauilion (~dcooley@c-71-198-86-127.hsd1.ca.comcast.net) Quit (Quit: leaving)
[19:04] * miroslav (~miroslav@c-98-248-210-170.hsd1.ca.comcast.net) has joined #ceph
[19:06] <barryo1> I've not touched cephfs yet, I'm just trying to figure out how to recreate the mds
[19:06] <barryo1> i found docs for removing and adding osds and mons, but not mds'
[19:06] * miroslav (~miroslav@c-98-248-210-170.hsd1.ca.comcast.net) Quit ()
[19:07] <dmick> barryo1: you can read mkcephfs's code for how it's done
[19:08] <scuttlemonkey> This may help: http://ceph.com/docs/master/man/8/ceph-mon/
[19:08] * Cube (~Cube@12.248.40.138) Quit (Quit: Leaving.)
[19:08] <dmick> scuttlemonkey: mds, not mon
[19:08] <scuttlemonkey> barryo1 ^
[19:08] <scuttlemonkey> oops
[19:08] <scuttlemonkey> wrong link
[19:09] * Cube (~Cube@12.248.40.138) has joined #ceph
[19:09] <scuttlemonkey> http://ceph.com/docs/master/man/8/mkcephfs/
[19:10] <scuttlemonkey> there we go, synergy decided it didn't want to cooperate there for a min
[19:10] <barryo1> thanks, i suppose the man page is probably a good place to start ;)
[19:11] * Vjarjadian (~IceChat77@5ad6d005.bb.sky.com) Quit (Read error: Connection reset by peer)
[19:11] * tziOm (~bjornar@ti0099a340-dhcp0628.bb.online.no) has joined #ceph
[19:11] <scuttlemonkey> barryo1: definitely...if you have more questions though, we're here
[19:11] <scuttlemonkey> (I hate getting rtfm + silence)
[19:14] * mcclurmc_laptop (~mcclurmc@firewall.ctxuk.citrix.com) Quit (Ping timeout: 480 seconds)
[19:15] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[19:16] * nick5 (~nick@74.222.153.12) has joined #ceph
[19:16] <barryo1> so mkcephfs --init-local-daemons mds -d /var/lib/ceph/mds/ceph-a should do the job?
[19:17] * nick5 (~nick@74.222.153.12) Quit (Remote host closed the connection)
[19:17] * nick5 (~nick@74.222.153.12) has joined #ceph
[19:18] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) Quit (Quit: tryggvil)
[19:19] <barryo1> i'm getting "global_init: unable to open config file from search list /var/lib/ceph/mds/ceph-a/conf"
[19:19] * rhar (~rhar@199.101.130.66) has joined #ceph
[19:19] * noahmehl (~noahmehl@cpe-75-186-45-161.cinci.res.rr.com) Quit (Quit: noahmehl)
[19:20] <rhar> i was thinking about using ceph on top of AWS EC2 instances with ephemeral storage. good idea? bad idea? I spent a couple minutes googling for AWS use case documentation but didn't see anything. Does anyone have pointers?
[19:21] <dmick> barryo1: -d is not about the data directory, I don't think, but rather a temporary directory
[19:22] <dmick> it needs to hold the monmap
[19:22] <dmick> I'm not sure how you get the monmap for mkcephfs itself, and it may or may not be easy to use mkcephfs for this purpose; I was suggesting reading the code to see how to invoke ceph-mds directly
[19:23] <dmick> I think it's something like "run ceph-mds with --mkfs (and other args), and then run it for reals"
[19:23] <scuttlemonkey> rhar: I do much of my deployment testing in ec2, but I don't know how it would stand up to production use
[19:23] <scuttlemonkey> I think it would largely depend on your use case
[19:24] * janos- (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[19:24] <scuttlemonkey> in fact, my juju deployment walkthrough was on ec2
[19:24] <scuttlemonkey> http://ceph.com/dev-notes/deploying-ceph-with-juju/
[19:24] <rhar> thanks I'll take a look.
[19:28] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) Quit (Quit: ZNC - http://znc.in)
[19:28] * janos- is now known as janos
[19:30] * nhorman (~nhorman@hmsreliant.think-freely.org) has joined #ceph
[19:34] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) Quit (Quit: ZNC - http://znc.in)
[19:35] * janos (janos@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[19:35] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[19:39] <mgalkiewicz> If I understand correctly upgrade from bobtail to 0.58 is only with downtime because 0.58 mons cannot be mixed with old ones. Can anyone confirm this?
[19:40] * Kioob (~kioob@2a01:e35:2432:58a0:21a:92ff:fe90:42c5) Quit (Quit: Leaving.)
[19:40] * fghaas (~florian@91-119-65-118.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[19:41] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[19:45] * ninkotech (~duplo@ip-89-102-24-167.net.upcbroadband.cz) has joined #ceph
[19:55] * jmlowe (~Adium@c-71-201-31-207.hsd1.in.comcast.net) Quit (Quit: Leaving.)
[19:56] * fghaas (~florian@91-119-65-118.dynamic.xdsl-line.inode.at) has joined #ceph
[19:57] <mjevans> mgalkiewicz: check the links in the topic
[19:58] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[19:59] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[20:02] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) has joined #ceph
[20:03] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[20:03] * __jt___ (~james@rhyolite.bx.mathcs.emory.edu) Quit (Ping timeout: 480 seconds)
[20:04] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Remote host closed the connection)
[20:07] <mjevans> mgalkiewicz: from the last link: New Ceph Monitor Changes http://ow.ly/ixgQN " So, no, upgrading is not complex, granted you are coming from Bobtail; otherwise, you will have to upgrade to Bobtail and take it from there."
[20:09] * sleinen (~Adium@2001:620:0:26:cc01:4338:bda5:c776) has joined #ceph
[20:10] <mgalkiewicz> I have read this and it states that we cannot mix mons
[20:16] <mjevans> Pretty much
[20:16] <mjevans> You'd have to take a given mon offline and bring it back in the new version, I think you'd be done upgrading when you have a redundant quorum on the new version.
[20:17] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) Quit (Ping timeout: 480 seconds)
[20:17] <mjevans> I'm not presently planning an upgrade, but I'd setup a test network first.
[20:17] <mgalkiewicz> I have staging cluster
[20:18] <mgalkiewicz> well it is to inconvenient I just wait for 0.56.4
[20:18] <iggy> rhar: we're considering that as well
[20:21] <rhar> iggy: have you done any of the experimentation yet?
[20:21] <rhar> like performance tests, randomly dropping nodes etc...
[20:22] <iggy> we tested gluster (not me, a previous admin) using ebs... it was unusable
[20:22] <iggy> my idea is to test with ephemeral storage like you were saying
[20:23] <iggy> we have some things to work out first (clusters spread across AZs, etc.)
[20:23] <iggy> so we haven't started testing yet
[20:24] <iggy> and actually, I'm waiting for my new laptop to get here before I start doing too much (I just started here)
[20:25] <iggy> thoughts on cross-AZ ceph deployment?
[20:26] <iggy> we're trying to decide whether we put ceph all in the same AZ and mount x-AZ or do we deploy ceph nodes in different AZs and hope that the x-AZ traffic doesn't kill us
[20:29] * The_Bishop (~bishop@2001:470:50b6:0:900f:3de0:4e76:c02f) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[20:30] * sleinen (~Adium@2001:620:0:26:cc01:4338:bda5:c776) Quit (Ping timeout: 480 seconds)
[20:31] * loicd1 (~loic@AAnnecy-257-1-103-124.w90-36.abo.wanadoo.fr) Quit (Quit: Leaving.)
[20:33] * nwat (~Adium@eduroam-242-170.ucsc.edu) has joined #ceph
[20:33] * sleinen (~Adium@2001:620:0:26:5810:919f:2510:a976) has joined #ceph
[20:39] <nwat> C++11 features a no-no?
[20:39] <joshd> yeah, not supported on many platforms yet
[20:40] <dmick> apparently the compilers are not yet ubiquitous, sadly
[20:40] <gregaf> soooo close…..*cry*
[20:40] <nwat> dang. that'd be nice. i really like using the lambda expression feature
[20:40] * miroslav (~miroslav@c-98-234-186-68.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[20:41] <dmick> autotyped iterators and 'foreach' make me drool.
[20:42] * oddover (~oddover@glados.colorado.edu) has joined #ceph
[20:43] <oddover> Hi. I'm trying to set up ceph, and I'm having trouble. "ceph -s" gives me "ERROR: missing keyring, cannot use cephx for authentication"
[20:44] <scuttlemonkey> hey oddover: taking the next step from the quickstart to multiple machines?
[20:44] <oddover> nope. still working on one :P
[20:45] <oddover> I'm just working really slow
[20:45] <scuttlemonkey> ahh
[20:45] <scuttlemonkey> so did you start over?
[20:45] <oddover> I think so.
[20:45] <oddover> I removed /var/lib/ceph, and started over. not sure if that's all I needed to do to start from scratch
[20:45] <scuttlemonkey> last I remember you were having issues trying to mount using cephfs locally
[20:46] <oddover> I left /etc/ceph and it's contents.
[20:46] <scuttlemonkey> hm
[20:46] * LeaChim (~LeaChim@b01bdcc0.bb.sky.com) Quit (Ping timeout: 480 seconds)
[20:46] <oddover> oh, no I had troubles mounting ceph from another machine
[20:46] <oddover> using cephfs
[20:47] <scuttlemonkey> well if all you did was blow away /var/lib/ceph the server is probably still expecting the original key
[20:48] <oddover> so what do I need to do to "start over"
[20:48] <scuttlemonkey> stop the service and uninstall ceph if you really wanna clean things out
[20:49] * LeaChim (~LeaChim@b01bdcc0.bb.sky.com) has joined #ceph
[20:49] <oddover> yea. I think I'll skip the reinstall if I can.
[20:49] <oddover> but I should rm /etc/ceph/ceph.keyring?
[20:49] <scuttlemonkey> do you have a ceph.client.admin.keyring?
[20:49] <oddover> no
[20:50] <scuttlemonkey> any .keyring files in /etc/ceph?
[20:50] <oddover> yea. ceph.keyring
[20:50] <scuttlemonkey> if you cat that file is it [client.admin] ?
[20:51] <oddover> yep
[20:51] <scuttlemonkey> and if you 'sudo ceph -s' you get a keyring error?
[20:51] * eschnou (~eschnou@148.91-201-80.adsl-dyn.isp.belgacom.be) has joined #ceph
[20:51] * sleinen (~Adium@2001:620:0:26:5810:919f:2510:a976) Quit (Quit: Leaving.)
[20:51] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[20:53] <oddover> nope
[20:54] <scuttlemonkey> ok...so it sounds like everything is fine?
[20:54] <oddover> that side looks like it
[20:54] <oddover> now I'm trying to do "sudo mount -t ceph {ip}:6789:/ /mnt/mycephfs"
[20:55] <scuttlemonkey> if you want to add a key for another user you can follow: http://ceph.com/docs/master/rados/operations/authentication/
[20:55] <oddover> and I get "mount: {ip}:6789:/: can't read superblock "
[20:55] <oddover> ({ip} is the ip of the host that I did the ceph -s, etc on)
[20:55] <scuttlemonkey> does that client machine have a keyring?
[20:56] <oddover> yep. and the didn't match. I just made them the same.
[20:57] <oddover> lemme see if that fixed it
[20:57] <oddover> nope
[20:58] * gaveen (~gaveen@182.161.25.135) Quit (Quit: Leaving)
[20:59] <scuttlemonkey> can you pastebin the output from ceph -s ?
[21:01] <oddover> http://pastebin.com/iPBqH70g
[21:02] * dosaboy (~gizmo@faun.canonical.com) Quit (Quit: Leaving.)
[21:03] <scuttlemonkey> can we take a look at what's in /var/lib/ceph/mds in the way of logs?
[21:03] <scuttlemonkey> curious if the mds is having trouble talking to the mons
[21:03] <scuttlemonkey> there is an up-to-date ceph.conf on the mds box, right?
[21:04] <oddover> yes. and the only thing in /var/lib/ceph/mds/ is ./ceph-a/keyring
[21:06] * leseb (~leseb@78.250.72.136) has joined #ceph
[21:06] <scuttlemonkey> can you kill the mds daemon and restart it with -f flag so we can see what's happening?
[21:06] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) has joined #ceph
[21:06] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) Quit ()
[21:06] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) has joined #ceph
[21:06] <scuttlemonkey> excuse me... -d
[21:06] * nhorman (~nhorman@hmsreliant.think-freely.org) Quit (Quit: Leaving)
[21:07] <scuttlemonkey> (debug mode)
[21:07] * loicd (~loic@AAnnecy-257-1-103-124.w90-36.abo.wanadoo.fr) has joined #ceph
[21:07] <oddover> "sudo service ceph -d -a start"?
[21:07] <scuttlemonkey> should be able to use ceph-mds
[21:07] <scuttlemonkey> http://ceph.com/docs/master/man/8/ceph-mds/
[21:10] <oddover> "sudo ceph-mds -d"?
[21:10] <oddover> sorry if I'm being dense. :P
[21:11] <dmick> you'll want to start it with the same flags it was started with
[21:11] <dmick> but adding the -d
[21:11] <scuttlemonkey> ^
[21:11] <scuttlemonkey> dmick: you good to help him out? I have to run out the door in about 5 mins here
[21:11] <dmick> I can try
[21:11] <scuttlemonkey> gracias
[21:12] <oddover> thanks dmick
[21:12] <oddover> I'm not sure I understand.
[21:12] <oddover> I've never used ceph-mds before
[21:12] <dmick> service ceph would have started it for you
[21:13] <dmick> what we're saying is to stop it manually and start it manually, but add -d
[21:13] <oddover> gotcha
[21:13] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) has joined #ceph
[21:14] <dmick> so it was probably run with --cluster=ceph -i a -f (assuming it was named mds.a)
[21:14] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[21:15] <dmick> you can also just let it start through 'service', and then look at its log, and increase the log level if it doesn't have much in it
[21:16] <oddover> "sudo /usr/bin/ceph-mds -d -i a --pid-file /var/run/ceph/mds.a.pid -c /etc/ceph/ceph.conf" should be sufficient, right?
[21:16] <dmick> seems right; pid-file is overkill but wno't hurt
[21:17] <oddover> and then does that put it in the foreground?
[21:17] <dmick> man ceph-mds will answer that, but yes
[21:17] <oddover> ok. sorry
[21:18] <dmick> (just gentle encouragement, no worries)
[21:18] * rturk-away is now known as rturk
[21:19] * mgalkiewicz (~mgalkiewi@staticline-31-182-128-35.toya.net.pl) Quit (Quit: Ex-Chat)
[21:19] * loicd (~loic@AAnnecy-257-1-103-124.w90-36.abo.wanadoo.fr) Quit (Quit: Leaving.)
[21:20] <oddover> ok, and then I do the mount command from my other host?
[21:20] <oddover> ord you just want the output from ceph-mds?
[21:20] <oddover> or do*
[21:22] <dmick> we're looking for clues about the failing operation by observing the daemon that should be answering
[21:23] <dmick> that means causing the failure while watching
[21:23] <oddover> ok. I tried the mount command, and didn't get anything on my other window.
[21:23] * LeaChim (~LeaChim@b01bdcc0.bb.sky.com) Quit (Ping timeout: 480 seconds)
[21:23] <dmick> well foo.
[21:23] <oddover> lol
[21:23] <dmick> is anything appearing in the mds log file?
[21:25] <oddover> what's the path for that?
[21:27] * serverascode (~serverasc@wwdyn182.cs.ualberta.ca) has joined #ceph
[21:27] <fghaas> /var/log/ceph/<cluster>-mds.<id>.log
[21:27] <dmick> if you haven't set it in ceph.conf...what fghaas said
[21:27] <fghaas> where <cluster> is normally ceph
[21:28] <oddover> ok, I don't have that file
[21:28] <dmick> ?
[21:28] <dmick> that's odd
[21:28] <oddover> scratch that
[21:28] <oddover> misread
[21:29] <oddover> I read it as "/var/lib..." lol *doh*
[21:30] <oddover> ok, didn't get anything in the log, but I did get something in ceph-mds
[21:30] <oddover> I'll pastebin it
[21:31] <oddover> http://pastebin.com/WcGjDrVP
[21:33] * markbby1 (~Adium@168.94.245.5) has joined #ceph
[21:33] * LeaChim (~LeaChim@b0fabd91.bb.sky.com) has joined #ceph
[21:33] * markbby (~Adium@168.94.245.2) Quit (Remote host closed the connection)
[21:33] * markbby (~Adium@168.94.245.2) has joined #ceph
[21:34] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) has joined #ceph
[21:34] * markbby1 (~Adium@168.94.245.5) Quit (Remote host closed the connection)
[21:35] * sleinen1 (~Adium@2001:620:0:26:9589:ed38:7d77:cbff) has joined #ceph
[21:36] <oddover> get anything from that?
[21:37] <dmick> finally loaded
[21:37] <dmick> sorry
[21:38] * stackevil (~stackevil@178.112.19.126.wireless.dyn.drei.com) has joined #ceph
[21:38] <dmick> no, not much.
[21:38] <oddover> np
[21:38] <oddover> didn't think so
[21:38] <dmick> so you're issuing sudo mount -t ceph {ip}:6789:/ /mnt/mycephfs
[21:39] <oddover> yep
[21:39] <oddover> from a different host
[21:39] * loicd (~loic@AAnnecy-257-1-103-124.w90-36.abo.wanadoo.fr) has joined #ceph
[21:40] <dmick> so you're not supplying a secret there
[21:41] <dmick> do you have authentication on? (you'd have to have disabled it explicitly)
[21:41] <dmick> http://ceph.com/docs/master/cephfs/kernel/
[21:42] <oddover> yes, I have it on then
[21:42] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[21:42] <dmick> so see the second and third usages there
[21:42] <oddover> yep
[21:42] <oddover> so would that be the same as the keyring key from before?
[21:43] <dmick> not sure what you did before, but you need the client.admin key
[21:43] <dmick> which will be stored in some keyring file
[21:43] <oddover> yep
[21:43] <dmick> and either supply it on the command line or put it in its own file (mount.ceph can't parse keyfiles, sadly)
[21:43] <dmick> s/keyfiles/keyring files/
[21:44] <oddover> gotcha
[21:44] <oddover> what's the "name" there?
[21:44] <dmick> the name of the client connection. clients are named client.name
[21:44] <dmick> the default is admin for most things. I'm not sure if -o name is actually required but it doesn't hurt
[21:45] <dmick> you can conceivably have multiple client names with different keys and thus access perms
[21:45] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) Quit (Quit: tryggvil)
[21:45] <oddover> gotcha
[21:45] <oddover> I'm retrying now with those options
[21:45] <oddover> still nothing
[21:46] <dmick> what was the mount command you used? (elide the key if you wish)
[21:47] <oddover> "sudo mount -t ceph <ip>:6789:/ /mnt/mycephfs -o name=admin,secret=<blah>"
[21:50] <dmick> verify that <blah> matches what you get out from ceph auth list
[21:50] <oddover> yep
[21:51] <dmick> ok, well, we'll need more mds logging, probably
[21:51] <dmick> you can stop that foregrounded one
[21:51] <dmick> http://ceph.com/docs/master/rados/configuration/ceph-conf/#ceph-logging-and-debugging will tell you general things about how to increase logging
[21:52] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[21:52] <dmick> I would start with [mds] debug mds = 20 and [global] debug auth = 20
[21:53] <oddover> k
[21:55] <oddover> I take it it's going to keep point out a lot of stuff to stdout?
[21:55] <oddover> it looks like the same stuff over and over
[21:56] <dmick> it may be coming to stderr for some things
[21:56] <dmick> (I assume you mean the mount command)
[21:56] <dmick> but the good stuff will be in the log(s)
[21:56] <dmick> either the mds log or the client.admin log
[21:57] <oddover> the ceph-mds command has lots of output
[21:57] <oddover> the mount one has almost nothing
[21:58] <dmick> "the ceph-mds command"? did you start it with -d again?
[21:58] <oddover> yep
[21:59] <dmick> oh. this was going to be ok to do through service. but that's all right
[21:59] * gucki (~smuxi@vpn.simplificator.com) Quit (Ping timeout: 480 seconds)
[21:59] * gucki (~smuxi@HSI-KBW-078-042-029-115.hsi3.kabel-badenwuerttemberg.de) has joined #ceph
[21:59] <dmick> let's check out the mds log and the client.admin log
[21:59] <oddover> mds log doesn't have anything
[21:59] <oddover> where's client.admin log?
[22:00] <dmick> same tree
[22:01] <dmick> did you restart the mds after you made those config changes?
[22:01] <oddover> yea
[22:01] <dmick> and did you put the config changes on the machine with the mds as well as the machine with the client?
[22:01] <oddover> er
[22:01] <oddover> no
[22:01] <oddover> no to that too
[22:01] <oddover> lemme do that
[22:01] <dmick> k
[22:05] * fghaas (~florian@91-119-65-118.dynamic.xdsl-line.inode.at) has left #ceph
[22:08] * eschnou (~eschnou@148.91-201-80.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[22:10] <barryo1> on an unrelated mds note, i fixed the issues i reported earlier by running service ceph -a stop mds && ceph mds rm a && ceph mds newfs 1 0 --yes-i-really-mean-it && service ceph -a start mds
[22:10] <dmick> happy to hear it
[22:10] <barryo1> i think that's what fixed it anyway
[22:11] <barryo1> there were a few other commands in between that lot
[22:11] <oddover> dmick: did that, still no luck
[22:11] <dmick> so the mds log should have some clues now when you try the mount
[22:11] <barryo1> if i had
[22:13] <barryo1> if i only had 1 mds in a production system and was using cephfs, if i lost the contents of /var/lib/ceph/mds/ceph-a/ would i be able to recover my files?
[22:13] * themgt (~themgt@24-177-232-181.dhcp.gnvl.sc.charter.com) has joined #ceph
[22:14] <barryo1> from cephfs i mean not the files in /var/lib/ceph/mds/ceph-a/
[22:14] <dmick> not certain. The metadata itself is stored in the cluster, not in /var/lib/ceph...but I don't know if a new mds would be able to cope with just the metadata. gregaf would know
[22:15] <oddover> dmick: still not getting anything in mds log
[22:15] <gregaf> huhwhat?
[22:15] <gregaf> oh, yeah, the only thing that lives in /var/lib/ceph/mds/ceph-a is going to be the keyring for the MDS, and maybe a conf file
[22:16] <dmick> oddover: not getting *anything*?
[22:16] <dmick> that is odd
[22:16] <oddover> nothing after I start up the service
[22:16] <barryo1> cool, thats good to know
[22:16] <gregaf> dmick: barryo: the MDSes don't keep anything locally except what they require in order to communicate with the monitor; if you lose it just replace it or set up a whole new identity
[22:17] <barryo1> are there many advantages to running more than one mds?
[22:17] <dmick> gregaf: thanks for confirming.
[22:18] <dmick> barryo1: live failover works, I think. Eventually you can also share load across multiple active mdses, by design, but that's not quite baked yet last I heard
[22:18] <dmick> so for right now "one active, multiple standbys" is the useful config AFAIK
[22:19] * diegows (~diegows@190.190.2.126) Quit (Ping timeout: 480 seconds)
[22:19] <iggy> there's a little more info on a recent blog post
[22:19] <iggy> of what's working now, what's expected to work for the first "stable" release of cephfs, what gets put off till later, etc.
[22:19] <oddover> dmick: I'm gonna start completely over (uninstall ceph, etc)
[22:21] <barryo1> iggy: is this the post http://ceph.com/dev-notes/cephfs-mds-status-discussion/?
[22:21] <dmick> oddover: ok. the next step would have been to add debug ms = 1 to everyone, because you certainly should be getting *some* logging, and messenger debug happens all the time
[22:21] <iggy> that the one by greg?
[22:21] <iggy> if so, yeah
[22:21] <barryo1> it's by gfarnum
[22:21] <iggy> yeah
[22:21] <dmick> aka gregaf :)
[22:21] <iggy> that's the one
[22:22] <barryo1> i'll read over that tomorrow, thanks
[22:23] * markbby (~Adium@168.94.245.2) Quit (Quit: Leaving.)
[22:28] * dspano (~dspano@rrcs-24-103-221-202.nys.biz.rr.com) has joined #ceph
[22:32] * The_Bishop (~bishop@2001:470:50b6:0:4d1c:295f:2468:e24a) has joined #ceph
[22:42] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) has joined #ceph
[22:51] * nwat (~Adium@eduroam-242-170.ucsc.edu) Quit (Quit: Leaving.)
[22:52] * serverascode (~serverasc@wwdyn182.cs.ualberta.ca) Quit (Ping timeout: 480 seconds)
[22:55] <rhar> we're in one az but eventually were moving over to cassandra which will solve our cross az issues.
[22:55] * BillK (~BillK@58-7-215-114.dyn.iinet.net.au) has joined #ceph
[22:55] * rhar (~rhar@199.101.130.66) Quit (Quit: Leaving)
[22:58] * dspano (~dspano@rrcs-24-103-221-202.nys.biz.rr.com) Quit (Quit: Leaving)
[23:02] * yanzheng (~zhyan@jfdmzpr06-ext.jf.intel.com) has joined #ceph
[23:03] * stackevil (~stackevil@178.112.19.126.wireless.dyn.drei.com) Quit (Quit: There are 10 types of people. Those who understand binary and those who don't.)
[23:05] * mcclurmc_laptop (~mcclurmc@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) has joined #ceph
[23:09] * __jt__ (~james@rhyolite.bx.mathcs.emory.edu) has joined #ceph
[23:15] * houkouonchi-work (~linux@12.248.40.138) Quit (Read error: Connection reset by peer)
[23:15] * houkouonchi-work (~linux@12.248.40.138) has joined #ceph
[23:17] * themgt (~themgt@24-177-232-181.dhcp.gnvl.sc.charter.com) Quit (Quit: themgt)
[23:19] * vata (~vata@2607:fad8:4:6:ce8:b0fc:4942:63ce) Quit (Quit: Leaving.)
[23:20] * themgt (~themgt@24-177-232-181.dhcp.gnvl.sc.charter.com) has joined #ceph
[23:21] * aliguori (~anthony@32.97.110.51) Quit (Remote host closed the connection)
[23:22] * dosaboy (~user1@host86-163-9-154.range86-163.btcentralplus.com) has joined #ceph
[23:22] * gucki (~smuxi@HSI-KBW-078-042-029-115.hsi3.kabel-badenwuerttemberg.de) Quit (Ping timeout: 480 seconds)
[23:27] * markbby (~Adium@168.94.245.6) has joined #ceph
[23:28] * nwat (~Adium@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[23:29] * nwat1 (~Adium@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[23:31] * markbby (~Adium@168.94.245.6) Quit ()
[23:33] * nwat1 (~Adium@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[23:35] <oddover> what ports do I need open for ceph?
[23:35] <oddover> right now I'm trying to mount cephfs as a fuse fs
[23:36] * xiaoxi (~xiaoxiche@134.134.137.73) has joined #ceph
[23:36] * nwat (~Adium@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[23:37] * stackevil (~stackevil@77.116.87.235.wireless.dyn.drei.com) has joined #ceph
[23:38] <iggy> ouch, and you shouldn't need any incoming ports open on the client (that I know of)
[23:38] <dmick> http://ceph.com/docs/master/rados/configuration/network-config-ref/#ip-tables
[23:42] * lofejndif (~lsqavnbok@bolobolo2.torservers.net) has joined #ceph
[23:43] * jtangwk (~Adium@2001:770:10:500:20fc:ee30:1f20:e4e2) Quit (Read error: Connection reset by peer)
[23:43] * jtangwk (~Adium@2001:770:10:500:c4bd:4063:193a:8bdd) has joined #ceph
[23:44] * mcclurmc_laptop (~mcclurmc@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) Quit (Read error: Operation timed out)
[23:44] * xiaoxi (~xiaoxiche@134.134.137.73) Quit (Ping timeout: 480 seconds)
[23:52] * PerlStalker (~PerlStalk@72.166.192.70) Quit (Quit: ...)
[23:57] * ScOut3R (~ScOut3R@c83-249-233-227.bredband.comhem.se) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.