#ceph IRC Log

Index

IRC Log for 2014-04-12

Timestamps are in GMT/BST.

[0:00] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[0:03] * zjohnson (~zjohnson@guava.jsy.net) has left #ceph
[0:07] * portante (~portante@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[0:07] * portante (~portante@nat-pool-bos-t.redhat.com) has joined #ceph
[0:10] * BillK (~BillK-OFT@106-69-75-112.dyn.iinet.net.au) has joined #ceph
[0:14] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[0:15] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[0:21] * alphe (~alphe@0001ac6f.user.oftc.net) has joined #ceph
[0:23] <alphe> hello all I have a problem with one osd at startup it doesnt mount the disk and so doesn t run
[0:23] <alphe> i tryed with ceph-disk-activate --make-init startup etc it doesn t work
[0:28] * ircolle (~Adium@2601:1:8380:2d9:e92e:7d80:9122:d3dc) Quit (Ping timeout: 480 seconds)
[0:28] * ircolle (~Adium@2601:1:8380:2d9:e92e:7d80:9122:d3dc) has joined #ceph
[0:36] * Boltsky (~textual@cpe-172-250-14-51.socal.res.rr.com) has joined #ceph
[0:39] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[0:44] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[0:44] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) Quit (Ping timeout: 480 seconds)
[0:50] * sputnik13 (~sputnik13@207.8.121.241) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[0:55] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) Quit (Remote host closed the connection)
[0:56] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Quit: ...)
[0:56] * diegows (~diegows@190.190.5.238) Quit (Ping timeout: 480 seconds)
[1:08] * ivotron (~ivotron@dhcp-59-213.cse.ucsc.edu) Quit (Read error: Operation timed out)
[1:13] * alram (~alram@38.122.20.226) Quit (Quit: leaving)
[1:14] * Boltsky (~textual@cpe-172-250-14-51.socal.res.rr.com) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[1:16] * dmsimard (~dmsimard@cloud.dmsimard.com) has joined #ceph
[1:32] * cynosure (~vilobhmm@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Quit: cynosure)
[1:33] * cynosure (~vilobhmm@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[1:34] * Boltsky (~textual@office.deviantart.net) has joined #ceph
[1:41] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[1:44] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[1:47] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[1:49] * xarses (~andreww@12.164.168.117) Quit (Ping timeout: 480 seconds)
[1:56] * rmoe (~quassel@12.164.168.117) Quit (Read error: Operation timed out)
[1:58] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[1:59] * ghartz (~ghartz@ip-68.net-80-236-84.joinville.rev.numericable.fr) Quit (Remote host closed the connection)
[2:06] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) has joined #ceph
[2:06] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) Quit (Quit: Leaving.)
[2:19] * ircolle (~Adium@2601:1:8380:2d9:e92e:7d80:9122:d3dc) Quit (Quit: Leaving.)
[2:22] * ircolle (~Adium@2601:1:8380:2d9:e92e:7d80:9122:d3dc) has joined #ceph
[2:29] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[2:29] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[2:31] * ircolle (~Adium@2601:1:8380:2d9:e92e:7d80:9122:d3dc) Quit (Quit: Leaving.)
[2:33] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[2:34] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[2:39] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[2:42] * sjustwork (~sam@2607:f298:a:607:6474:f09d:d6f3:18fa) Quit (Quit: Leaving.)
[2:45] * Cube (~Cube@66.87.65.145) Quit (Quit: Leaving.)
[2:59] * alphe (~alphe@0001ac6f.user.oftc.net) Quit (Quit: Leaving)
[3:03] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[3:05] * davidzlap (~Adium@ip68-5-239-214.oc.oc.cox.net) Quit (Quit: Leaving.)
[3:08] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[3:16] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[3:32] * masterpe (~masterpe@2a01:670:400::43) Quit (Read error: Connection reset by peer)
[3:36] * masterpe (~masterpe@2a01:670:400::43) has joined #ceph
[4:09] * The_Bishop_ (~bishop@f055103169.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[4:12] * yuriw (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[4:13] * yuriw (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) has joined #ceph
[4:13] * Shmouel1 (~Sam@fny94-12-83-157-27-95.fbx.proxad.net) has joined #ceph
[4:18] * Shmouel (~Sam@fny94-12-83-157-27-95.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[4:18] * zack_dolby (~textual@p852cae.tokynt01.ap.so-net.ne.jp) has joined #ceph
[4:29] * JCL1 (~JCL@2601:9:5980:39b:81d0:318b:8102:2fc9) has joined #ceph
[4:29] * JCL (~JCL@c-24-23-166-139.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[4:40] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[4:45] * zerick (~eocrospom@190.187.21.53) Quit (Remote host closed the connection)
[5:15] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has joined #ceph
[5:16] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) has joined #ceph
[5:16] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has left #ceph
[5:23] * cynosure (~vilobhmm@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Quit: cynosure)
[5:35] * Vacum_ (~vovo@i59F79BA0.versanet.de) has joined #ceph
[5:40] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Remote host closed the connection)
[5:41] * fdmanana_ (~fdmanana@bl10-141-238.dsl.telepac.pt) has joined #ceph
[5:42] * Vacum (~vovo@i59F798FF.versanet.de) Quit (Ping timeout: 480 seconds)
[5:49] * fdmanana (~fdmanana@bl13-136-121.dsl.telepac.pt) Quit (Ping timeout: 480 seconds)
[6:01] * wgao (~wgao@1.202.252.122) has joined #ceph
[6:01] * jlogan (~Thunderbi@2600:c00:3010:1:1::40) Quit (Quit: jlogan)
[6:17] * MACscr (~Adium@c-50-158-183-38.hsd1.il.comcast.net) Quit (Quit: Leaving.)
[6:25] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[6:25] * ChanServ sets mode +v andreask
[6:25] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[6:32] * Boltsky (~textual@office.deviantart.net) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[6:39] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[6:59] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[7:00] * Boltsky (~textual@cpe-198-72-138-106.socal.res.rr.com) has joined #ceph
[7:01] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) has joined #ceph
[7:15] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[7:17] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[7:25] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[7:25] * ChanServ sets mode +v andreask
[7:26] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[7:38] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[7:49] * tserong (~tserong@203-57-208-132.dyn.iinet.net.au) Quit (Quit: Leaving)
[7:50] * tserong (~tserong@203-214-57-114.dyn.iinet.net.au) has joined #ceph
[8:05] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[8:14] * cynosure (~vilobhmm@c-50-152-188-98.hsd1.ca.comcast.net) has joined #ceph
[8:14] * Cataglottism (~Cataglott@dsl-087-195-030-184.solcon.nl) has joined #ceph
[8:15] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) Quit (Quit: Leaving.)
[8:19] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[8:25] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[8:44] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[8:46] * Cube (~Cube@66.87.131.182) has joined #ceph
[8:46] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[8:47] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[8:52] * madkiss1 (~madkiss@089144226206.atnat0035.highway.a1.net) has joined #ceph
[8:55] * cynosure_ (~vilobhmm@nat-dip6.cfw-a-gci.corp.yahoo.com) has joined #ceph
[8:55] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[8:57] * Svedrin (svedrin@ketos.funzt-halt.net) Quit (Quit: Starved on the internet)
[8:57] * Cube (~Cube@66.87.131.182) Quit (Ping timeout: 480 seconds)
[8:58] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[8:59] * Svedrin (svedrin@ketos.funzt-halt.net) has joined #ceph
[8:59] * cynosure (~vilobhmm@c-50-152-188-98.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[8:59] * cynosure_ is now known as cynosure
[9:08] * Cataglottism (~Cataglott@dsl-087-195-030-184.solcon.nl) Quit (Quit: My Mac Pro has gone to sleep. ZZZzzz???)
[9:10] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) has joined #ceph
[9:14] * AfC (~andrew@2001:44b8:31cb:d400:2ad2:44ff:fe08:a4c) has joined #ceph
[9:18] * wido (~wido@2a00:f10:121:100:4a5:76ff:fe00:199) Quit (Quit: No Ping reply in 180 seconds.)
[9:20] * wido (~wido@2a00:f10:121:100:4a5:76ff:fe00:199) has joined #ceph
[9:25] * wido (~wido@2a00:f10:121:100:4a5:76ff:fe00:199) Quit (Quit: No Ping reply in 180 seconds.)
[9:25] * wido (~wido@2a00:f10:121:100:4a5:76ff:fe00:199) has joined #ceph
[9:29] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[9:29] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[9:39] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[9:40] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) has joined #ceph
[9:41] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[9:41] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[9:46] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit ()
[9:48] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[9:49] * pvh_sa (~pvh@197.87.135.34) has joined #ceph
[9:50] <pvh_sa> hey there... i've got 5 clients that need to share a filesystem - 4 of them are also going to be block stores for the fs - is cephfs mature enough to use for such a scenario? last time i tried cephfs it was pretty unstable
[9:51] * sz0 (~user@94.55.194.230) has joined #ceph
[9:52] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[9:52] * cowbar (~cow@ip-2607-F298-0001-0100-0000-0000-0000-FFFF.dreamhost.com) Quit (Quit: leaving)
[9:53] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[9:54] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[9:54] * cowbar (~cow@ip-2607-F298-0001-0100-0000-0000-0000-FFFF.dreamhost.com) has joined #ceph
[9:58] * cowbar (~cow@ip-2607-F298-0001-0100-0000-0000-0000-FFFF.dreamhost.com) Quit ()
[10:00] * cowbar (~cow@ip-2607-F298-0001-0100-0000-0000-0000-FFFF.dreamhost.com) has joined #ceph
[10:21] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) has joined #ceph
[10:23] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[10:23] * ChanServ sets mode +v andreask
[10:24] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) Quit (Quit: My Mac Pro has gone to sleep. ZZZzzz???)
[10:31] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) Quit (Ping timeout: 480 seconds)
[10:40] * MACscr (~Adium@c-50-158-183-38.hsd1.il.comcast.net) has joined #ceph
[10:45] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[10:46] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[10:48] * cynosure (~vilobhmm@nat-dip6.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[10:53] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) has joined #ceph
[10:56] * sz0 (~user@94.55.194.230) Quit (Quit: ERC Version 5.3 (IRC client for Emacs))
[11:16] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[11:24] * sz0 (~sz0@94.55.194.230) has joined #ceph
[11:25] * tserong_ (~tserong@124-171-119-127.dyn.iinet.net.au) has joined #ceph
[11:30] * tserong (~tserong@203-214-57-114.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[11:31] * BillK (~BillK-OFT@106-69-75-112.dyn.iinet.net.au) Quit (Quit: ZNC - http://znc.in)
[11:33] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[11:37] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[11:39] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[11:40] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[11:48] * humbolt (~elias@chello080109074153.4.15.vie.surfer.at) has joined #ceph
[12:06] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[12:12] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[12:24] * tserong__ (~tserong@124-171-114-133.dyn.iinet.net.au) has joined #ceph
[12:28] * tserong_ (~tserong@124-171-119-127.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[12:32] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has joined #ceph
[12:32] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has left #ceph
[12:36] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) has joined #ceph
[12:41] * ghartz (~ghartz@ip-68.net-80-236-84.joinville.rev.numericable.fr) has joined #ceph
[12:41] * BillK (~BillK-OFT@106-69-75-112.dyn.iinet.net.au) has joined #ceph
[13:19] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[13:21] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[13:25] * sz0 is now known as sz0`
[13:26] * sz0` is now known as sz0
[13:32] * maybe (proxy@bilbo.bmiag.de) has joined #ceph
[13:33] <- *maybe* Hi
[13:33] <- *maybe* I have a big problem with a ceph-cluster.
[13:33] <- *maybe* Is anybody here that can help me?
[13:34] <maybe> Hi. I got a big problem with a ceph cluster.
[13:34] <maybe> My monitors are not coming up.
[13:35] <maybe> Anybody capable of helping me?
[13:36] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[13:49] * sz0 is now known as sz0`
[13:53] <maybe> Anybody can help me with a problem?
[13:55] <cowbar> may want to wait a few hours
[13:56] <cowbar> quite a few of the ceph guys are PST, and its 5am here
[13:56] <cowbar> I'd help but I don't much much/any direct exp with ceph
[14:00] <maybe> yeah well, thx
[14:00] <maybe> it 14:00 here in middle europe
[14:00] <maybe> i thought many people might be sleeping
[14:01] <maybe> but i am quite in panic :(
[14:01] <kraken> http://i.imgur.com/fH9e2.gif
[14:06] <classicsnail> are they crashing, or are they not getting quorum?
[14:06] <maybe> they are not getting quorum
[14:06] <maybe> they always fall back to "probing"
[14:07] <maybe> sometime stuck in "electing" atm they are stuck in "probing"
[14:08] <maybe> i am trying to get 2 of 3 running for now, one is switched off
[14:10] <maybe> i am gone a while to fix me a meal
[14:10] <maybe> will come back later
[14:10] <maybe> maybe more people are up then
[14:10] <classicsnail> tried the things in http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/ ?
[14:10] <maybe> yes
[14:10] <maybe> as far as i can tell
[14:10] <classicsnail> not a firewall between hosts?
[14:10] <maybe> no
[14:10] <maybe> i had troubles with the clocks
[14:10] <maybe> so i fixed that now
[14:11] <maybe> they are in sync below < 0.01
[14:11] <classicsnail> not sure I'd jump straight to it, but you could potentially modify the monmap and drop it back to one monitor, add the others back in once it's up
[14:12] <maybe> that idea came to me too, but i never did that before
[14:12] <maybe> the map is the same it seems btw
[14:12] <maybe> so i am not sure if that will help
[14:13] <classicsnail> probing usually means it's trying to find and talk to the other monitors for quorum
[14:13] <maybe> they all say c is 0, a is 1 and b is 2
[14:13] <maybe> yeah
[14:13] <maybe> i figured they might be having trouble connecting
[14:13] <maybe> but telnet to the monitor ports works
[14:14] <maybe> so i rule out connection troubles
[14:14] <maybe> c says c is outside quorum and b says b is outside quorum and a is stopped
[14:14] <maybe> they will start with election if i restart one
[14:15] <maybe> for a short time c gets to be the leader
[14:15] <maybe> then they fall back to probing
[14:15] <classicsnail> you don't have stateful firewalling on linux or something?
[14:15] <classicsnail> I've had the conntrack table get full, and cause the monitors to fail
[14:16] <maybe> I have seen similar too, but these three hosts all have completely empty firewall rules
[14:17] <classicsnail> my next step in an emergency would be to choose one of the hosts, and modify the monmap so it is the only monitor
[14:17] <classicsnail> then repair the other monitors
[14:17] <classicsnail> by recreating them
[14:17] <maybe> i dont now how to change the monmap yet
[14:17] <maybe> and i dont know which monitor would be best to choose
[14:17] <maybe> can you help me?
[14:18] <classicsnail> it's in that document
[14:18] <maybe> then i didnt read good enough, sorry
[14:18] <maybe> i will try
[14:18] <classicsnail> actually, sorry, not in that exact page
[14:19] <classicsnail> http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#removing-monitors
[14:19] <classicsnail> the unhealthy cluster steps
[14:19] <maybe> oh well
[14:19] <maybe> i see
[14:19] <maybe> Removing Monitors from an Unhealthy Cluster ?
[14:19] <classicsnail> I've done that quite a few times on a test cluster, and once in production when it all went pear shaped
[14:19] <classicsnail> YMMV and the rest
[14:20] <maybe> it okay, but i dont know which monitor to choose
[14:20] <maybe> which is dead? which is healthy? they all probe....
[14:20] * madkiss1 (~madkiss@089144226206.atnat0035.highway.a1.net) Quit (Quit: Leaving.)
[14:20] <classicsnail> you have to shut them all down
[14:20] <classicsnail> choose one
[14:20] <classicsnail> which one you choose, I can't answer that for you unfortunately
[14:21] <maybe> okay
[14:21] <maybe> i take c, that was leader for a bit
[14:21] <classicsnail> read through that document very carefully
[14:22] <classicsnail> I'm not a ceph expert, just some random guy on the internet who has had those instructions work for him
[14:22] <maybe> might be a good idea to make a backup of that monitor first
[14:23] <classicsnail> definitely
[14:26] <maybe> he is up and he is leader
[14:32] <classicsnail> join the osds and mds if you have them, see if your data is valid and the cluster is working
[14:39] <maybe> osds wont start
[14:40] <maybe> looks better now
[14:40] <classicsnail> so the osds are joining the cluster?
[14:41] <maybe> some do, some dont (timeout)
[14:41] <classicsnail> could be load related?
[14:41] <classicsnail> as in, the cluster is doing a lot of repair and peering?
[14:41] <maybe> its a very slow machine that got the problems
[14:43] <maybe> most osds are up
[14:44] <maybe> one machine gets timeout
[14:45] <maybe> thank you so far!
[14:48] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[14:51] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[14:51] * madkiss1 (~madkiss@178.188.60.118) has joined #ceph
[14:56] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[14:57] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit ()
[14:59] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[15:01] <maybe> Now i got: osd.1 33155 unable to obtain rotating service keys; retrying
[15:01] <maybe> who obtains those keys from whom?
[15:02] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[15:02] <classicsnail> I don't know sorry
[15:02] * thanhtran (~thanhtran@113.172.130.81) has joined #ceph
[15:02] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[15:03] * madkiss1 (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[15:04] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has joined #ceph
[15:05] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has left #ceph
[15:06] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[15:06] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[15:08] * sz0` is now known as sz0
[15:10] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Ping timeout: 480 seconds)
[15:21] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) has joined #ceph
[15:23] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[15:25] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[15:29] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[15:29] * AfC (~andrew@2001:44b8:31cb:d400:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[15:31] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[15:32] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[15:46] * Daviey (~DavieyOFT@bootie.daviey.com) Quit (Ping timeout: 480 seconds)
[15:51] * thanhtran_ (~thanhtran@113.172.130.81) has joined #ceph
[15:52] * nhm (~nhm@184-97-144-70.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[15:53] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[15:56] * sz0 (~sz0@94.55.194.230) Quit ()
[15:58] * thanhtran (~thanhtran@113.172.130.81) Quit (Ping timeout: 480 seconds)
[15:58] * thanhtran_ is now known as thanhtran
[15:58] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[15:59] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[16:01] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[16:02] * LeaChim (~LeaChim@host86-162-1-71.range86-162.btcentralplus.com) has joined #ceph
[16:04] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[16:05] * thanhtran (~thanhtran@113.172.130.81) Quit (Quit: Going offline, see ya! (www.adiirc.com))
[16:05] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[16:10] * LeaChim (~LeaChim@host86-162-1-71.range86-162.btcentralplus.com) Quit (Remote host closed the connection)
[16:27] * _sileht (~sileht@gizmo.sileht.net) has joined #ceph
[16:29] * BillK (~BillK-OFT@106-69-75-112.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[16:29] * sileht (~sileht@gizmo.sileht.net) Quit (Ping timeout: 480 seconds)
[16:31] * analbeard (~shw@host86-155-192-138.range86-155.btcentralplus.com) has joined #ceph
[16:36] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[16:36] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[16:39] * analbeard (~shw@host86-155-192-138.range86-155.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[16:41] * Daviey (~DavieyOFT@87.117.230.170) has joined #ceph
[16:46] * oblu (~o@62.109.134.112) Quit (Quit: ~)
[16:47] * oblu (~o@62.109.134.112) has joined #ceph
[16:48] * analbeard (~shw@support.memset.com) has joined #ceph
[16:53] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[16:58] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[16:58] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[16:59] * ghartz (~ghartz@ip-68.net-80-236-84.joinville.rev.numericable.fr) Quit (Remote host closed the connection)
[17:03] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[17:03] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[17:04] * jbd (~jbd@2001:41d0:52:a00::77) has joined #ceph
[17:14] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[17:22] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) has joined #ceph
[17:22] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[17:23] * jbd (~jbd@2001:41d0:52:a00::77) has left #ceph
[17:26] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[17:30] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[17:50] * scuttlemonkey (~scuttlemo@c-107-5-193-244.hsd1.mi.comcast.net) has joined #ceph
[17:50] * ChanServ sets mode +o scuttlemonkey
[17:51] * pvh_sa (~pvh@197.87.135.34) Quit (Read error: Operation timed out)
[17:52] * pvh_sa (~pvh@197.87.135.34) has joined #ceph
[17:54] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[18:01] * tserong__ (~tserong@124-171-114-133.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[18:02] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[18:03] * tserong__ (~tserong@58-6-130-172.dyn.iinet.net.au) has joined #ceph
[18:07] * sz0 (~sz0@94.55.194.230) has joined #ceph
[18:27] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[18:38] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[18:52] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[18:53] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[18:55] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[18:56] * Zethrok (~martin@95.154.26.34) Quit (Ping timeout: 480 seconds)
[18:58] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[18:58] * tserong_ (~tserong@124-168-224-82.dyn.iinet.net.au) has joined #ceph
[18:59] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[19:03] * tserong__ (~tserong@58-6-130-172.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[19:04] * lollipop (~s51itxsyc@23.94.38.19) has joined #ceph
[19:06] <lollipop> http://paste.fedoraproject.org/93780/13973223/
[19:07] <lollipop> ^^^ this is from compute.log on an openstack compute node.
[19:08] <lollipop> while "rbd -p volumes ls" in command line says good.
[19:08] <lollipop> cinder creation/deletion both works well
[19:08] <lollipop> any hint for this error?
[19:10] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[19:10] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) has joined #ceph
[19:11] * lollipop (~s51itxsyc@23.94.38.19) Quit (Quit: Lost terminal)
[19:14] * lollipop (~s51itxsyc@23.94.38.19) has joined #ceph
[19:23] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) has joined #ceph
[19:29] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[19:31] * allsystemsarego (~allsystem@188.25.132.17) has joined #ceph
[19:31] * sjustlaptop (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) has joined #ceph
[19:32] * sjustlaptop (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) has left #ceph
[19:44] * rotbeard (~redbeard@2a02:908:df10:6f00:76f0:6dff:fe3b:994d) has joined #ceph
[19:49] * madkiss (~madkiss@178.188.60.118) Quit (Quit: Leaving.)
[19:49] * valtha (~valtha@ohmu.fi) has joined #ceph
[20:07] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) has joined #ceph
[20:10] * robbat2 (~robbat2@2001:470:e889:1:212:32ff:fe00:221e) has left #ceph
[20:23] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) has joined #ceph
[20:38] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[21:09] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) has joined #ceph
[21:21] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[21:24] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) has joined #ceph
[21:25] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) Quit (Remote host closed the connection)
[21:25] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) has joined #ceph
[21:26] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[21:31] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Ping timeout: 480 seconds)
[21:32] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[21:50] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) Quit (Quit: My Mac Pro has gone to sleep. ZZZzzz???)
[21:54] * oro (~oro@77-59-135-139.dclient.hispeed.ch) has joined #ceph
[21:54] * oro (~oro@77-59-135-139.dclient.hispeed.ch) Quit ()
[22:00] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) has joined #ceph
[22:05] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[22:13] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[22:19] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has joined #ceph
[22:20] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has left #ceph
[22:25] * mschiff (~mschiff@mx10.schiffbauer.net) Quit (Remote host closed the connection)
[22:25] * mschiff (~mschiff@mx10.schiffbauer.net) has joined #ceph
[22:32] * Osmann (~numlock@88.234.55.201) has joined #ceph
[22:32] * Osmann (~numlock@88.234.55.201) Quit (autokilled: Do not spam other people. Mail support@oftc.net if you feel this is in error. (2014-04-12 20:32:35))
[22:36] * cynosure (~vilobhmm@c-50-152-188-98.hsd1.ca.comcast.net) has joined #ceph
[22:48] * Cube1 (~Cube@66.87.131.182) has joined #ceph
[22:48] * Cube (~Cube@66-87-131-182.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[22:50] * mnash (~chatzilla@66-194-114-178.static.twtelecom.net) Quit (Ping timeout: 480 seconds)
[22:51] * simulx2 (~simulx@vpn.expressionanalysis.com) Quit (Ping timeout: 480 seconds)
[22:55] * simulx (~simulx@66-194-114-178.static.twtelecom.net) has joined #ceph
[23:03] * Cube1 (~Cube@66.87.131.182) Quit (Quit: Leaving.)
[23:12] * cynosure (~vilobhmm@c-50-152-188-98.hsd1.ca.comcast.net) Quit (Quit: cynosure)
[23:12] * analbeard (~shw@host86-155-192-138.range86-155.btcentralplus.com) has joined #ceph
[23:17] * mnash (~chatzilla@vpn.expressionanalysis.com) has joined #ceph
[23:19] * theactualwarrenusui (~Warren@2607:f298:a:607:dcc9:4a1b:2f10:9fe0) Quit (Read error: Connection reset by peer)
[23:19] * theactualwarrenusui (~Warren@2607:f298:a:607:dcc9:4a1b:2f10:9fe0) has joined #ceph
[23:25] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) has joined #ceph
[23:33] * ivotron (~ivotron@c-24-5-147-181.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[23:49] * topro (~prousa@host-62-245-142-50.customer.m-online.net) Quit (Ping timeout: 480 seconds)
[23:50] * fdmanana_ (~fdmanana@bl10-141-238.dsl.telepac.pt) Quit (Quit: Leaving)
[23:55] * analbeard (~shw@host86-155-192-138.range86-155.btcentralplus.com) Quit (Quit: Leaving.)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.