#ceph IRC Log

Index

IRC Log for 2014-02-24

Timestamps are in GMT/BST.

[0:02] <bdonnahue2> im having trouble with rpm --import https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc
[0:07] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[0:19] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[0:19] * themgt (~themgt@pc-146-48-120-200.cm.vtr.net) has joined #ceph
[0:22] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[0:25] <bdonnahue2> fixed the issue
[0:25] <bdonnahue2> firewall blocking https
[0:32] * themgt (~themgt@pc-146-48-120-200.cm.vtr.net) Quit (Quit: Pogoapp - http://www.pogoapp.com)
[0:36] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) has joined #ceph
[0:38] * codice (~toodles@97-94-175-73.static.mtpk.ca.charter.com) Quit (Ping timeout: 480 seconds)
[0:39] * fatih (~fatih@c-50-174-71-251.hsd1.ca.comcast.net) has joined #ceph
[0:40] * sjm (~sjm@cpe-72-225-145-68.nj.res.rr.com) has joined #ceph
[0:46] <bdonnahue2> can ceph run on a 32 bit machine?
[0:48] <sage> bdonnahue2: yes
[0:48] <sage> everything but the ceph-fuse (fs client)
[0:49] * codice (~toodles@97-94-175-73.static.mtpk.ca.charter.com) has joined #ceph
[0:53] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[1:10] * Midnightmyth_ (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[1:23] <bdonnahue2> im seeing this error:
[1:23] <bdonnahue2> RuntimeError: /etc/ceph/ does not exist - could not write config
[1:23] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:24] <bdonnahue2> while running ceph-deploy mon create-initial
[1:29] <aarontc> bdonnahue2: did you use ceph-deploy to install ceph? if not, you might have to 'mkdir /etc/ceph' yourself
[1:29] <aarontc> I see some pretty awesome stats bugs sometimes... 41791 MB/s rd, 68811 MB/s wr, 556 kop/s; 3137688/10710696 objects degraded (29.295%); 105 GB/s, 27244 objects/s recoverin
[1:37] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[1:38] <bdonnahue2> aarontc yes i did
[1:38] <bdonnahue2> i vererted my vms and am starting the process again
[1:41] <aarontc> bdonnahue2: hm, well I don't use ceph-deploy but I know you can get into a weird state by doing a clean or purge or something
[1:42] <aarontc> I'm having a problem right now myself that I can't get my monitors to start
[1:45] <aarontc> two of my mons are saying "numerical limit out of domain" when I try to launch them
[1:49] <aarontc> appears to be crashing in a list iterator copy method called from (OSDMap::decode_classic(ceph::buffer::list::iterator&)+0x426) [0x7cbe26]
[2:00] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[2:01] * glzhao (~glzhao@220.181.11.232) has joined #ceph
[2:02] * sjm (~sjm@cpe-72-225-145-68.nj.res.rr.com) Quit (Remote host closed the connection)
[2:03] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[2:07] * ivotron_ (~ivotron@adsl-99-146-3-213.dsl.pltn13.sbcglobal.net) has joined #ceph
[2:07] * ivotron (~ivotron@adsl-99-146-3-213.dsl.pltn13.sbcglobal.net) Quit (Read error: Connection reset by peer)
[2:09] * yanzheng (~zhyan@134.134.137.73) has joined #ceph
[2:11] <aarontc> is there any data that would be useful to anyone about the two crashing mons before I blow them away and create new ones?
[2:11] <aarontc> I got 3 of 5 to come up after upgrading to 0.77
[2:15] <sage> aarontc: can you fpaste the end of the log?
[2:16] <aarontc> sage: http://hastebin.com/pavodolobi
[2:17] * LeaChim (~LeaChim@host86-166-182-74.range86-166.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:19] <sage> aarontc: upgrading *to* 0.77, right?
[2:19] <sage> from which version?
[2:19] <aarontc> sage: correct, from 0.76
[2:20] <aarontc> I shut down all the OSDs and mons, did the upgrade, then started all 5 mons again and only 3 came back
[2:21] <sage> would you mind sending a copy of the mon data dir for one of the crashing ones?
[2:21] <aarontc> not at all
[2:24] <aarontc> sage: http://www.aarontc.com/ceph-mon-janeway-for-sage.tar.bz2
[2:25] <aarontc> I'll preserve the data from the other mon as well, in case that'll be useful. I want to get the last two mons online again so I have some redundancy
[2:37] <aarontc> I also have an issue that I was about to dig into logging to track down - there are 127pgs "down", even though both acting OSDs are up
[2:38] <aarontc> if there's a quick answer to that and someone wants to share, I'm open to suggestions :)
[2:39] * geekmush1 (~Adium@cpe-66-68-198-33.rgv.res.rr.com) has joined #ceph
[2:45] * geekmush (~Adium@cpe-66-68-198-33.rgv.res.rr.com) Quit (Ping timeout: 480 seconds)
[2:46] <sage> aarontc: got it, looking now
[2:48] * shimo (~A13032@122x212x216x66.ap122.ftth.ucom.ne.jp) Quit (Quit: shimo)
[2:48] * shimo (~A13032@122x212x216x66.ap122.ftth.ucom.ne.jp) has joined #ceph
[2:55] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[2:57] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) Quit (Quit: doppelgrau)
[2:57] * kaizh_ (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) has joined #ceph
[3:01] * shang (~ShangWu@175.41.48.77) has joined #ceph
[3:04] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[3:07] * ingard (~cake@tu.rd.vc) Quit (Ping timeout: 480 seconds)
[3:15] <sage> aarontc: cool, i found the bug.
[3:15] <sage> thanks!
[3:15] <sage> i think you'll just need to blow away that monitor and re-add it.
[3:17] <aarontc> sage: glad I could help, and blowing it away and adding again seems to have solved the issue :)
[3:18] <aarontc> I also figured out why my pgs are down, they are waiting for a down OSD to come back apparently -- "blocked": "peering is blocked due to down osds",
[3:18] <aarontc> (even though the osd to be probed isn't listed in the acting set)
[3:18] * shellcmd (~quassel@static-50-53-102-132.bvtn.or.frontiernet.net) Quit (Ping timeout: 480 seconds)
[3:21] * ivotron_ (~ivotron@adsl-99-146-3-213.dsl.pltn13.sbcglobal.net) Quit (Ping timeout: 480 seconds)
[3:22] * ivotron (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) has joined #ceph
[3:23] <sage> aarontc: what name/email should i use for Reported-by:?
[3:23] <aarontc> sage: Aaron Ten Clay <aarontc@aarontc.com> is fine
[3:23] <sage> thanks!
[3:23] <aarontc> you did all the hard work, lol ;)
[3:25] * erkules (~erkules@port-92-193-65-219.dynamic.qsc.de) has joined #ceph
[3:27] <sage> btw don't forget to take down that tarball
[3:28] <aarontc> will do, thanks
[3:32] * erkules_ (~erkules@port-92-193-120-53.dynamic.qsc.de) Quit (Ping timeout: 480 seconds)
[3:43] * haomaiwang (~haomaiwan@106.38.255.123) Quit (Remote host closed the connection)
[3:44] * haomaiwang (~haomaiwan@113.196.160.69) has joined #ceph
[3:49] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[3:50] * sarob (~sarob@2601:9:7080:13a:e4a3:7be5:3894:67c2) has joined #ceph
[3:51] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[3:54] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[3:58] * sarob (~sarob@2601:9:7080:13a:e4a3:7be5:3894:67c2) Quit (Ping timeout: 480 seconds)
[3:59] * haomaiwa_ (~haomaiwan@113.196.160.69) has joined #ceph
[3:59] * haomaiwang (~haomaiwan@113.196.160.69) Quit (Read error: Connection reset by peer)
[3:59] * Nats_ (~Nats@telstr575.lnk.telstra.net) Quit (Read error: Connection reset by peer)
[4:02] * sglwlb (~sglwlb@221.12.27.202) Quit ()
[4:07] * Nats (~Nats@telstr575.lnk.telstra.net) has joined #ceph
[4:08] * haomaiwang (~haomaiwan@118.186.133.131) has joined #ceph
[4:09] * ingard (~cake@tu.rd.vc) has joined #ceph
[4:15] * haomaiwa_ (~haomaiwan@113.196.160.69) Quit (Ping timeout: 480 seconds)
[4:17] * sarob (~sarob@2601:9:7080:13a:bdc6:848e:c781:46c6) has joined #ceph
[4:17] * sarob (~sarob@2601:9:7080:13a:bdc6:848e:c781:46c6) Quit (Remote host closed the connection)
[4:17] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[4:22] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[4:22] * ingard (~cake@tu.rd.vc) Quit (Ping timeout: 480 seconds)
[4:23] * haomaiwang (~haomaiwan@118.186.133.131) Quit (Ping timeout: 480 seconds)
[4:24] * haomaiwang (~haomaiwan@117.79.232.213) has joined #ceph
[4:30] * glzhao (~glzhao@220.181.11.232) Quit (Ping timeout: 480 seconds)
[4:33] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[4:35] * bandrus (~Adium@66-87-126-141.pools.spcsdns.net) has joined #ceph
[4:38] * glzhao (~glzhao@220.181.11.232) has joined #ceph
[4:38] * ivotron (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) Quit (Read error: Connection reset by peer)
[4:38] * ivotron (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) has joined #ceph
[4:40] * ivotron_ (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) has joined #ceph
[4:40] * ivotron (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) Quit (Read error: Connection reset by peer)
[4:41] * kaizh_ (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[4:46] * glzhao (~glzhao@220.181.11.232) Quit (Ping timeout: 480 seconds)
[4:46] * glzhao_ (~glzhao@220.181.11.232) has joined #ceph
[4:52] * glzhao (~glzhao@220.181.11.232) has joined #ceph
[4:54] * ivotron_ (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) Quit (Remote host closed the connection)
[4:54] * bandrus (~Adium@66-87-126-141.pools.spcsdns.net) Quit (Quit: Leaving.)
[4:54] * fatih (~fatih@c-50-174-71-251.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[4:55] * ivotron (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) has joined #ceph
[4:55] * glzhao_ (~glzhao@220.181.11.232) Quit (Ping timeout: 480 seconds)
[5:22] * glzhao_ (~glzhao@220.181.11.232) has joined #ceph
[5:22] * glzhao (~glzhao@220.181.11.232) Quit (Read error: Connection reset by peer)
[5:23] * Cube (~Cube@66-87-130-93.pools.spcsdns.net) Quit (Quit: Leaving.)
[5:36] * ingard (~cake@tu.rd.vc) has joined #ceph
[5:36] * Vacum (~vovo@i59F7AFCE.versanet.de) has joined #ceph
[5:42] * AfC (~andrew@2407:7800:400:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[5:43] * Vacum_ (~vovo@i59F7941A.versanet.de) Quit (Ping timeout: 480 seconds)
[5:47] * shellcmd (~quassel@static-50-53-102-132.bvtn.or.frontiernet.net) has joined #ceph
[5:49] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[5:49] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[5:51] * sarob_ (~sarob@2601:9:7080:13a:dd61:dbd2:97bf:b714) has joined #ceph
[5:54] * Cube (~Cube@66-87-130-93.pools.spcsdns.net) has joined #ceph
[5:57] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[6:02] * Cube (~Cube@66-87-130-93.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[6:03] * sarob_ (~sarob@2601:9:7080:13a:dd61:dbd2:97bf:b714) Quit (Ping timeout: 480 seconds)
[6:03] * AfC (~andrew@2407:7800:400:1011:2ad2:44ff:fe08:a4c) has joined #ceph
[6:08] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[6:11] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[6:11] * ivotron (~ivotron@adsl-99-146-2-252.dsl.pltn13.sbcglobal.net) Quit (Remote host closed the connection)
[6:22] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) has joined #ceph
[6:29] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[6:31] <bens> i like sundays and i cannot lay
[6:31] <bens> lie.
[6:35] * shellcmd (~quassel@static-50-53-102-132.bvtn.or.frontiernet.net) Quit (Ping timeout: 480 seconds)
[6:36] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[6:48] * Cube (~Cube@66-87-130-93.pools.spcsdns.net) has joined #ceph
[6:56] * Cube (~Cube@66-87-130-93.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[7:04] * sjusthm (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) Quit (Ping timeout: 480 seconds)
[7:16] * haomaiwa_ (~haomaiwan@117.79.232.213) has joined #ceph
[7:16] * haomaiwang (~haomaiwan@117.79.232.213) Quit (Read error: Connection reset by peer)
[7:17] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[7:18] * sarob (~sarob@2601:9:7080:13a:5434:5cf:e418:d338) has joined #ceph
[7:26] * sarob (~sarob@2601:9:7080:13a:5434:5cf:e418:d338) Quit (Ping timeout: 480 seconds)
[7:28] * mattt (~textual@94.236.7.190) has joined #ceph
[7:36] * AfC (~andrew@2407:7800:400:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[7:41] * mattt_ (~textual@92.52.76.140) has joined #ceph
[7:44] * mattt (~textual@94.236.7.190) Quit (Ping timeout: 480 seconds)
[7:44] * mattt_ is now known as mattt
[7:45] * b0e (~aledermue@juniper1.netways.de) has joined #ceph
[8:08] * srenatus (~stephan@e179112157.adsl.alicedsl.de) has joined #ceph
[8:08] * ghost (~deeppatel@122.160.123.34) has joined #ceph
[8:08] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) Quit (Quit: Leaving.)
[8:19] * ghost (~deeppatel@122.160.123.34) has left #ceph
[8:22] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) has joined #ceph
[8:25] <josef_> Have you guys been doing a lot of rdb mounts? I read that in 2.6+ it should be possible to do more than 255 mount (since minor numbers are 20bit value then), but that tools could have limits in themselves.
[8:25] <josef_> Is it feasible to do with rbd? i.e. having one mount per user on a server with 400 accounts?
[8:52] * Sysadmin88 (~IceChat77@176.254.32.31) Quit (Quit: We be chillin - IceChat style)
[8:57] * rendar (~s@host7-177-dynamic.20-87-r.retail.telecomitalia.it) has joined #ceph
[9:00] * srenatus (~stephan@e179112157.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[9:05] * srenatus (~stephan@e179112157.adsl.alicedsl.de) has joined #ceph
[9:09] * thb (~me@2a02:2028:6d:c7d0:6267:20ff:fec9:4e40) has joined #ceph
[9:10] * foosinn (~stefan@office.unitedcolo.de) has joined #ceph
[9:17] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[9:19] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[9:20] * srenatus (~stephan@e179112157.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[9:24] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Quit: Leaving.)
[9:26] * srenatus (~stephan@e179112157.adsl.alicedsl.de) has joined #ceph
[9:26] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[9:26] <sekon> Hello,
[9:26] <sekon> ceph health is giving me the following error
[9:26] <sekon> ceph health
[9:26] <sekon> Error initializing cluster client: Error
[9:27] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[9:27] <sekon> i dont get any more details in verbose mode and nothing is being appended to the ceph.log file in my ceph-deploy directory
[9:27] <sekon> how can i get more information on the error so that i can try to fix it
[9:28] * peetaur (~peter@x2f181f8.dyn.telefonica.de) has joined #ceph
[9:29] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[9:38] * srenatus (~stephan@e179112157.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[9:39] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[9:47] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[9:54] * srenatus (~stephan@185.27.182.2) has joined #ceph
[9:55] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[9:55] * ChanServ sets mode +v andreask
[9:57] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[10:01] * TMM (~hp@c97185.upc-c.chello.nl) Quit (Quit: Ex-Chat)
[10:01] * yanzheng (~zhyan@134.134.137.73) Quit (Quit: Leaving)
[10:19] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[10:27] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[10:29] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[10:31] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[10:32] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[10:39] * LeaChim (~LeaChim@host86-166-182-74.range86-166.btcentralplus.com) has joined #ceph
[10:48] * mattt (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[11:02] * allsystemsarego (~allsystem@188.25.129.255) has joined #ceph
[11:13] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[11:19] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[11:20] * mattt (~textual@94.236.7.190) has joined #ceph
[11:22] <zidarsk8> hi, can anyone help me a bit: I have 5 node ceph cluster, osd-s running on nodes 0 1 2 3 4, and monitors on nodes 1 2 3.
[11:22] <zidarsk8> if i run ceph -s on a osd node that's not a monitor, i get ceph status,
[11:22] <zidarsk8> but if i run that on a monitor node I get a keyring error
[11:22] <zidarsk8> $ ceph -s
[11:22] <zidarsk8> 2014-02-10 13:43:15.148849 7fc40c6a5700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentica
[11:23] * sarob (~sarob@2601:9:7080:13a:c5e1:9fae:555e:5029) has joined #ceph
[11:24] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[11:26] <andreask> zidarsk8: admin keyring/key is available and readable?
[11:29] <zidarsk8> damn ... that's a silly error. the permissions were 600
[11:30] <zidarsk8> thank you andreask ... but why would the permissions be wrong on a clean install ... and why are they different on monitor nodes
[11:31] <andreask> you installed all vie ceph-deploy?
[11:31] <zidarsk8> jup
[11:31] * sarob (~sarob@2601:9:7080:13a:c5e1:9fae:555e:5029) Quit (Ping timeout: 480 seconds)
[11:31] <zidarsk8> http://pastebin.com/7aPZ7bT0 - this was used to install it
[11:32] * BillK (~BillK-OFT@124-148-105-206.dyn.iinet.net.au) Quit (Quit: ZNC - http://znc.in)
[11:32] <andreask> komisch .. strange ;-)
[11:32] <zidarsk8> I have the whole install script and centos 6.5 images to retry it as many times as i like ... and it's always the same
[11:32] <andreask> admin key file is owned by root on all nodes?
[11:33] <zidarsk8> jes
[11:33] <zidarsk8> yes*
[11:34] <zidarsk8> the only difference i've seen is that on osd nodes the permissions are 644 and on monitors they are 600
[11:35] <andreask> and you run the ceph -s as non-root user?
[11:35] * Svedrin (svedrin@ketos.funzt-halt.net) Quit (Ping timeout: 480 seconds)
[11:36] <srenatus> hmm weird thing here. one node down, but the recovery is acting weirdly with "cephx: verify_authorizer could not get service secret for service osd secret_id=140" on many OSDs
[11:36] <zidarsk8> yes
[11:36] <zidarsk8> i have created a ceph user as in tutorial
[11:36] * mattt (~textual@94.236.7.190) Quit (Read error: Operation timed out)
[11:37] <zidarsk8> the ceph user has sudo rights and all (no password needed)
[11:38] <andreask> zidarsk8: hmm ... then I don't see a reason why permissions should be a problem if you du as sudo
[11:39] * mattt (~textual@92.52.76.140) has joined #ceph
[11:41] <zidarsk8> okay ... I just thought i should be able to run everything as taht ceph user. Thanks for the help andreask
[11:41] <andreask> yw
[11:45] <srenatus> how can I lookup a secret_id? I wonder what's the problem here
[11:45] <srenatus> (the % of degraded objs increases steadily...)
[11:48] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[12:02] <andreask> srenatus: hmm ... what is your ceph version?
[12:07] <madkiss1> loicd: You about? :)
[12:08] <loicd> yes, about to go to lunch though
[12:08] <loicd> madkiss1: what can I do for you today ?
[12:09] <srenatus> andreask: testing, 0.76..
[12:09] <madkiss1> loicd: have you seen http://www.meetup.com/OpenStack-MeetUp-Frankfurt/ ? :)
[12:11] <madkiss1> I think we're confronted with the fact that we now have an OpenStack Ceph Meetup done by HP at the HP venue and the first meeting of the Ceph Meetup Frankfurt at another place, both at the same time (the evening before the Ceph Day)
[12:11] <madkiss1> I am not sure that's the best thing ever
[12:12] * mmmucky (~mucky@mucky.socket7.org) Quit (Ping timeout: 480 seconds)
[12:13] * capri_on (~capri@212.218.127.222) has joined #ceph
[12:14] <srenatus> apparently, things just started repairing... weird, it took a while.
[12:15] * ZyTer (~ZyTer@ghostbusters.apinnet.fr) Quit (Ping timeout: 480 seconds)
[12:15] <srenatus> one pg is down+peering because both it's other replicas are on the node that has gone down
[12:15] <srenatus> how this has happened (two of three replica on _one_ host), no idea.
[12:15] <srenatus> what are the implications of one `pg` being down?
[12:16] <andreask> srenatus: you have a custom crush-map? the default is to spread the copies over hosts
[12:17] <srenatus> andreask: nothing custom yet
[12:17] * capri_oner (~capri@212.218.127.222) Quit (Read error: Operation timed out)
[12:18] <andreask> srenatus: can you pastebin a "ceph osd tree"?
[12:18] * ZyTer (~ZyTer@ghostbusters.apinnet.fr) has joined #ceph
[12:19] <srenatus> andreask: yep
[12:19] <srenatus> one moment
[12:20] <srenatus> andreask: https://gist.github.com/srenatus/0f2110136af478431adf
[12:21] <andreask> srenatus: oh ... that's all?
[12:21] <srenatus> uhmm. yes?
[12:22] <andreask> hmmm ... and your ceph.conf?
[12:22] * mmmucky (~mucky@mucky.socket7.org) has joined #ceph
[12:23] <srenatus> andreask: updated the gist
[12:23] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[12:23] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[12:26] * BillK (~BillK-OFT@124-148-105-206.dyn.iinet.net.au) has joined #ceph
[12:26] <andreask> srenatus: ah ... sorry, overlooked the interesting part before ... looks ok
[12:26] <andreask> srenatus: and "ceph osd crush rule dump"?
[12:28] <srenatus> andreask: updated
[12:31] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[12:32] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[12:32] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[12:34] * glzhao_ (~glzhao@220.181.11.232) Quit (Quit: leaving)
[12:35] <andreask> srenatus: looks ok
[12:45] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[12:56] <sekon> can anyone help me with my Error initializing cluster client: Error
[12:57] <sekon> I have redone the quick ceph deploy guide multiple times with the same result and i cant seem to find out what is wrong
[13:02] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[13:06] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) has joined #ceph
[13:06] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) Quit ()
[13:06] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) has joined #ceph
[13:07] * ircuser-1 (~ircuser-1@35.222-62-69.ftth.swbr.surewest.net) Quit (Read error: Operation timed out)
[13:23] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[13:23] * dmick (~dmick@2607:f298:a:607:8902:da2e:ba5f:224c) Quit (Ping timeout: 480 seconds)
[13:26] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[13:26] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[13:28] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) Quit (Quit: Leaving)
[13:29] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[13:30] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) has joined #ceph
[13:32] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) Quit ()
[13:32] <ccooke> hmm. is there any sane way to get ceph to store replicas on a remote OSD for DR/backup but not have that impact performance locally?
[13:32] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) has joined #ceph
[13:33] * dmick (~dmick@2607:f298:a:607:d999:cecb:1914:20ac) has joined #ceph
[13:34] <shang> hi all, I have a question about the federated gateway
[13:35] <shang> ideally, I want to setup three sites and have people access (read/write) data
[13:35] <shang> how should I architect it?
[13:35] <shang> I have read the: http://ceph.com/docs/master/radosgw/federated-config/
[13:36] * b0e (~aledermue@juniper1.netways.de) Quit (Ping timeout: 480 seconds)
[13:36] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[13:36] * shang (~ShangWu@175.41.48.77) Quit (Quit: Ex-Chat)
[13:37] <ccooke> ... *laugh* I should have read that part of the documentation, I guess
[13:40] * b0e (~aledermue@juniper1.netways.de) has joined #ceph
[13:42] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[13:54] * ircuser-1 (~ircuser-1@35.222-62-69.ftth.swbr.surewest.net) has joined #ceph
[13:58] <jerker> what is the status for ceph on zfs? fine?
[13:58] <jerker> http://wiki.ceph.com/Planning/Blueprints/Emperor/osd%3A_ceph_on_zfs
[14:01] * banks (~banks@host86-154-234-37.range86-154.btcentralplus.com) has joined #ceph
[14:01] * banks (~banks@host86-154-234-37.range86-154.btcentralplus.com) Quit ()
[14:06] <srenatus> we had the following situation today: one MON host had its hdd mounted read-only (due to a hardware failure probably), but the mon service did not die. just after we stopped the ceph-mon service did the other two MONs "take over" (I'm still lacking a complete understanding and thus probably use the wrong words...)
[14:06] <srenatus> where "the MONs do stuff" for me means "ceph health" returns anything different from a timeout
[14:07] <srenatus> so I wonder - how can you remedy this situation in the future? I.e. just die when things look weird instead of misbehaving
[14:25] * jcsp (~Adium@0001bf3a.user.oftc.net) has joined #ceph
[14:33] * BillK (~BillK-OFT@124-148-105-206.dyn.iinet.net.au) Quit (Quit: ZNC - http://znc.in)
[14:37] * Svedrin (svedrin@ketos.funzt-halt.net) has joined #ceph
[14:42] * BillK (~BillK-OFT@124-148-105-206.dyn.iinet.net.au) has joined #ceph
[14:44] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) has joined #ceph
[14:54] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[14:54] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[14:55] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[14:55] * b0e (~aledermue@juniper1.netways.de) Quit (Quit: Leaving.)
[15:00] * b0e (~aledermue@juniper1.netways.de) has joined #ceph
[15:14] * sroy (~sroy@207.96.182.162) has joined #ceph
[15:18] * erice (~erice@50.240.86.181) has joined #ceph
[15:19] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[15:20] <srenatus> uhmm can I use 0.77 packages with 0.76 packages?
[15:22] * BillK (~BillK-OFT@124-148-105-206.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[15:23] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[15:32] * gsaxena (~gsaxena@pool-108-56-185-35.washdc.fios.verizon.net) has joined #ceph
[15:33] <srenatus> nevermind. just won't try.
[15:38] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[15:39] * markbby (~Adium@168.94.245.1) has joined #ceph
[15:47] * yguang11 (~yguang11@vpn-nat.corp.tw1.yahoo.com) has joined #ceph
[15:48] * mnash (~chatzilla@66-194-114-178.static.twtelecom.net) Quit (Ping timeout: 480 seconds)
[15:50] * simulx2 (~simulx@66-194-114-178.static.twtelecom.net) Quit (Ping timeout: 480 seconds)
[15:52] * simulx (~simulx@66-194-114-178.static.twtelecom.net) has joined #ceph
[15:53] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) has joined #ceph
[15:59] * mnash (~chatzilla@vpn.expressionanalysis.com) has joined #ceph
[16:09] * b0e (~aledermue@juniper1.netways.de) Quit (Remote host closed the connection)
[16:11] * sboyette (~sboyette@50-199-109-158-static.hfc.comcastbusiness.net) has joined #ceph
[16:13] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Read error: Operation timed out)
[16:17] * shellcmd (~quassel@static-50-53-102-132.bvtn.or.frontiernet.net) has joined #ceph
[16:20] * linuxkidd (~linuxkidd@2001:420:2100:2258:39d3:de25:be2d:1e03) has joined #ceph
[16:31] <srenatus> if one OSD just won't join again (X up, X-1 in), what could be the case? logs look ok to me, https://gist.github.com/srenatus/4d148e78fc6079f73b94 , but what do I know...
[16:38] * sleinen (~Adium@2001:620:0:46:2526:fd8:337a:458a) has joined #ceph
[16:41] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) Quit (Quit: Leaving)
[16:47] * oro (~oro@2001:620:20:222:5de7:b047:62fc:c382) Quit (Ping timeout: 480 seconds)
[16:48] * xarses_ (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[16:53] * mattt (~textual@92.52.76.140) Quit (Ping timeout: 480 seconds)
[16:54] * yguang11 (~yguang11@vpn-nat.corp.tw1.yahoo.com) Quit (Ping timeout: 480 seconds)
[16:54] * mattt (~textual@92.52.76.140) has joined #ceph
[16:55] * angdraug (~angdraug@c-67-169-181-128.hsd1.ca.comcast.net) has joined #ceph
[17:00] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) has joined #ceph
[17:01] * abique (~abique@time2market1.epfl.ch) Quit (Ping timeout: 480 seconds)
[17:05] * abique (~abique@time2market1.epfl.ch) has joined #ceph
[17:05] * JeffK (~JeffK@38.99.52.10) Quit (Read error: Connection reset by peer)
[17:06] * JeffK (~JeffK@38.99.52.10) has joined #ceph
[17:09] * foosinn (~stefan@office.unitedcolo.de) Quit (Remote host closed the connection)
[17:15] * peetaur (~peter@x2f181f8.dyn.telefonica.de) Quit (Remote host closed the connection)
[17:16] * hasues (~hasues@kwfw01.scrippsnetworksinteractive.com) has joined #ceph
[17:17] * peetaur (~peter@x2f181f8.dyn.telefonica.de) has joined #ceph
[17:18] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[17:18] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[17:18] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[17:20] * xarses_ (~andreww@12.164.168.117) has joined #ceph
[17:22] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[17:23] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Read error: Connection reset by peer)
[17:25] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[17:28] <jerker> the installation procedures at http://ceph.com/docs/master/install/get-packages/ are in my opinion to manual. There should be a ceph-release package that includes a correct yum.repo.d file and then just go ahead and install. Compare with ZFS http://zfsonlinux.org/epel.html
[17:32] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[17:34] * sprachgenerator (~sprachgen@130.202.135.209) has joined #ceph
[17:35] <dmsimard> jerker: There are ways to deploy ceph. ceph-deploy, chef cookbooks, puppet recipes
[17:35] * peetaur is now known as Guest1102
[17:35] * peetaur (~peter@x2f181f8.dyn.telefonica.de) has joined #ceph
[17:35] * Guest1102 (~peter@x2f181f8.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[17:36] <alfredodeza> jerker: oh but there is a ceph-release package
[17:40] * thb (~me@0001bd58.user.oftc.net) Quit (Quit: Leaving.)
[17:45] * sjustwork (~sam@2607:f298:a:607:bd74:b4e:a47d:8123) has joined #ceph
[17:46] * sprachgenerator (~sprachgen@130.202.135.209) Quit (Quit: sprachgenerator)
[17:47] * sprachgenerator (~sprachgen@130.202.135.209) has joined #ceph
[17:51] * elmo (~james@faun.canonical.com) has joined #ceph
[17:54] * gregsfortytwo (~Adium@2607:f298:a:607:50e5:2de0:a5d4:604a) Quit (Quit: Leaving.)
[17:54] * gregsfortytwo (~Adium@2607:f298:a:607:9050:ab35:adc6:8d17) has joined #ceph
[17:55] * nwat (~textual@eduroam-246-164.ucsc.edu) has joined #ceph
[18:01] * ivotron (~ivotron@dhcp-59-237.cse.ucsc.edu) has joined #ceph
[18:03] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[18:04] * zerick (~eocrospom@190.187.21.53) has joined #ceph
[18:05] * allsystemsarego (~allsystem@188.25.129.255) Quit (Quit: Leaving)
[18:09] * ivotron (~ivotron@dhcp-59-237.cse.ucsc.edu) Quit (Ping timeout: 480 seconds)
[18:11] * ivotron (~ivotron@dhcp-59-237.cse.ucsc.edu) has joined #ceph
[18:11] * mattt (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[18:12] * zerick (~eocrospom@190.187.21.53) Quit (Quit: Saliendo)
[18:13] * wrencsok (~wrencsok@wsip-174-79-34-244.ph.ph.cox.net) Quit (Quit: Leaving.)
[18:13] * wrencsok (~wrencsok@wsip-174-79-34-244.ph.ph.cox.net) has joined #ceph
[18:14] * nwat (~textual@eduroam-246-164.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[18:14] * zerick (~eocrospom@190.187.21.53) has joined #ceph
[18:16] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) Quit (Quit: Leaving.)
[18:17] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[18:19] * allsystemsarego (~allsystem@188.25.129.255) has joined #ceph
[18:20] * reed (~reed@75-101-54-131.dsl.static.sonic.net) has joined #ceph
[18:21] * jcsp1 (~jcsp@109.144.239.185) has joined #ceph
[18:21] * jcsp1 (~jcsp@109.144.239.185) Quit (Remote host closed the connection)
[18:23] * nwat (~textual@eduroam-246-164.ucsc.edu) has joined #ceph
[18:26] * JCL (~JCL@2601:9:5980:39b:fcf8:8631:910f:bc02) has joined #ceph
[18:26] * JCL (~JCL@2601:9:5980:39b:fcf8:8631:910f:bc02) Quit (Remote host closed the connection)
[18:28] * JCL (~JCL@2601:9:5980:39b:fcf8:8631:910f:bc02) has joined #ceph
[18:33] * rmoe (~quassel@12.164.168.117) has joined #ceph
[18:34] * gsaxena (~gsaxena@pool-108-56-185-35.washdc.fios.verizon.net) Quit (Remote host closed the connection)
[18:38] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[18:43] * nwat (~textual@eduroam-246-164.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[18:44] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[18:53] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) has joined #ceph
[18:54] * janos_ (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[18:55] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[18:55] * srenatus (~stephan@185.27.182.2) Quit (Ping timeout: 480 seconds)
[18:58] * xmltok_ (~xmltok@216.103.134.250) has joined #ceph
[18:58] * rotbeard (~redbeard@2a02:908:df19:7a80:76f0:6dff:fe3b:994d) has joined #ceph
[19:01] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[19:02] * Cube (~Cube@12.248.40.138) has joined #ceph
[19:02] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[19:03] * angdraug (~angdraug@c-67-169-181-128.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[19:04] * janos_ (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) Quit (Quit: ZNC - http://znc.in)
[19:04] * janos_ (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[19:12] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) Quit (Quit: ZNC - http://znc.in)
[19:12] * janos_ is now known as janos
[19:13] * janos_ (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[19:13] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) Quit (Quit: neurodrone)
[19:13] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) has joined #ceph
[19:13] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) Quit (Quit: ZNC - http://znc.in)
[19:14] * janos_ is now known as janos
[19:14] * warrenSusui (~Warren@2607:f298:a:607:38fc:445b:1848:70d4) has joined #ceph
[19:16] <saturnine> Any suggested options for RBD backed VMs?
[19:16] <saturnine> e.g., rbd cache/filestore flusher settings, etc.
[19:16] * fghaas (~florian@85-127-219-50.dynamic.xdsl-line.inode.at) has joined #ceph
[19:18] * wsusui (~Warren@2607:f298:a:607:38fc:445b:1848:70d4) has joined #ceph
[19:19] * wsusui (~Warren@2607:f298:a:607:38fc:445b:1848:70d4) has left #ceph
[19:21] * xarses_ (~andreww@12.164.168.117) Quit (Remote host closed the connection)
[19:21] * wusui (~Warren@2607:f298:a:607:14c3:461e:7720:b8ec) Quit (Ping timeout: 480 seconds)
[19:21] * WarrenUsui (~Warren@2607:f298:a:607:14c3:461e:7720:b8ec) Quit (Ping timeout: 480 seconds)
[19:21] * wusui (~Warren@2607:f298:a:607:38fc:445b:1848:70d4) has joined #ceph
[19:23] * jcsp (~Adium@0001bf3a.user.oftc.net) Quit (Quit: Leaving.)
[19:24] * jcsp (~Adium@0001bf3a.user.oftc.net) has joined #ceph
[19:28] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[19:31] * xarses (~andreww@12.164.168.117) has joined #ceph
[19:33] * imjustmatthew (~imjustmat@pool-72-84-198-231.rcmdva.fios.verizon.net) Quit (Remote host closed the connection)
[19:36] * ircolle (~Adium@mobile-166-137-183-188.mycingular.net) has joined #ceph
[19:37] * angdraug (~angdraug@12.164.168.117) has joined #ceph
[19:39] * rudolfsteiner (~federicon@181.109.26.141) has joined #ceph
[19:39] * fghaas (~florian@85-127-219-50.dynamic.xdsl-line.inode.at) has left #ceph
[19:40] * rudolfsteiner (~federicon@181.109.26.141) Quit ()
[19:47] * joshuay04 (~joshuay04@rrcs-74-218-204-10.central.biz.rr.com) has joined #ceph
[19:48] <joshuay04> Hello, what might be the cause of "2014-02-24 12:43:59.874363 osd.3 [WRN] 10 slow requests, 5 included below; oldest blocked for > 31.910014 secs"
[19:48] <joshuay04> For no reason all of my hosts cpus are at 100%, no scrub is running and I am only at 200op/s
[19:52] * ircolle (~Adium@mobile-166-137-183-188.mycingular.net) Quit (Quit: Leaving.)
[19:56] * thb (~me@2a02:2028:6d:c7d0:6267:20ff:fec9:4e40) has joined #ceph
[19:56] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[19:56] * ircolle (~Adium@mobile-166-137-183-188.mycingular.net) has joined #ceph
[19:56] * thb is now known as Guest1190
[19:56] * ircolle1 (~Adium@210.193.201.205.brainstorminternet.net) has joined #ceph
[19:58] * ircolle (~Adium@mobile-166-137-183-188.mycingular.net) Quit (Read error: Connection reset by peer)
[19:59] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) Quit (Quit: ZNC - http://znc.in)
[20:08] <gregsfortytwo> joshuay04: I'd start with the "ceph -s" output; there's obviously something going on; maybe that'll give you a clue, or the OSD logs or admin socket, or looking at general system health
[20:09] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[20:12] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) has joined #ceph
[20:13] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:14] <joshuay04> gregsfortytwo: Thanks, what would cause the cpu to max out? All 5 hosts are at 100% yet there is no scrub going on. I have never seen this before
[20:15] <bens> is ceph taking the cpu?
[20:15] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) Quit ()
[20:16] * scuttlemonkey changes topic to 'Latest stable (v0.72.x "Emperor") -- http://ceph.com/get || dev channel #ceph-devel || Ceph is a GSoC 2014 mentoring org: http://goo.gl/dYqVAf'
[20:16] <scuttlemonkey> if anyone knows students that would like to get paid to work on Ceph this summer, send them my way
[20:16] <scuttlemonkey> will send out a more detailed update on list / social media in a bit
[20:16] <scuttlemonkey> hooray for Google Summer of Code
[20:17] <hasues> scuttlemonkey: URL me the details, and I'll ask around.
[20:18] <joshuay04> bens: Yes looking at htop the osd is taking the cpu
[20:18] <scuttlemonkey> hasues: http://www.google-melange.com/gsoc/org2/google/gsoc2014/ceph
[20:18] <scuttlemonkey> that links to our ideas page and has all of our various contact info
[20:18] <hasues> scuttlemonkey: Cool. thanks!
[20:18] <scuttlemonkey> or they can just hit patrick@inktank.com with questions
[20:18] <ircolle1> scuttlemonkey - can you copy this info to #ceph-devel ?
[20:18] <scuttlemonkey> np
[20:19] <scuttlemonkey> ircolle1: yeah
[20:19] <ircolle1> scuttlemonkey - thanks!
[20:22] * janos (~messy@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[20:22] * rudolfsteiner (~federicon@181.109.26.141) has joined #ceph
[20:30] * scalability-junk (uid6422@id-6422.ealing.irccloud.com) Quit ()
[20:31] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[20:31] * dmsimard1 (~Adium@70.38.0.246) has joined #ceph
[20:31] * srenatus (~stephan@e179112157.adsl.alicedsl.de) has joined #ceph
[20:32] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit ()
[20:36] * dmsimard (~Adium@108.163.152.2) Quit (Read error: Connection reset by peer)
[20:39] * xarses (~andreww@12.164.168.117) Quit (Quit: Leaving)
[20:41] * xarses (~andreww@12.164.168.117) has joined #ceph
[20:43] * dmsimard1 (~Adium@70.38.0.246) Quit (Quit: Leaving.)
[20:43] * dmsimard (~Adium@70.38.0.246) has joined #ceph
[20:45] * dmsimard1 (~Adium@palpatine.privatedns.com) has joined #ceph
[20:46] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[20:46] * ChanServ sets mode +v andreask
[20:46] * joshuay04 (~joshuay04@rrcs-74-218-204-10.central.biz.rr.com) Quit ()
[20:47] * ircolle1 (~Adium@210.193.201.205.brainstorminternet.net) Quit (Quit: Leaving.)
[20:49] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[20:49] * hasues (~hasues@kwfw01.scrippsnetworksinteractive.com) Quit (Quit: Leaving.)
[20:50] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit ()
[20:51] * dmsimard (~Adium@70.38.0.246) Quit (Ping timeout: 480 seconds)
[20:52] * ircolle (~Adium@210.193.201.205.brainstorminternet.net) has joined #ceph
[20:56] * fatih (~fatih@208.72.139.54) has joined #ceph
[20:58] * rudolfsteiner (~federicon@181.109.26.141) Quit (Quit: rudolfsteiner)
[21:01] * ircolle (~Adium@210.193.201.205.brainstorminternet.net) Quit (Ping timeout: 480 seconds)
[21:01] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[21:08] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) Quit (Quit: neurodrone)
[21:08] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) has joined #ceph
[21:09] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) Quit ()
[21:09] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) has joined #ceph
[21:20] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[21:20] * ChanServ sets mode +v andreask
[21:30] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[21:31] * dmsimard1 (~Adium@palpatine.privatedns.com) Quit (Read error: Operation timed out)
[21:37] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) has joined #ceph
[21:47] * hasues (~hasues@kwfw01.scrippsnetworksinteractive.com) has joined #ceph
[21:49] * peetaur is now known as Guest1270
[21:49] * peetaur (~peter@x2f181f8.dyn.telefonica.de) has joined #ceph
[21:51] * srenatus (~stephan@e179112157.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[21:52] * Sysadmin88 (~IceChat77@176.254.32.31) has joined #ceph
[21:54] * Guest1270 (~peter@x2f181f8.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[21:56] * sroy (~sroy@207.96.182.162) Quit (Quit: Quitte)
[21:58] * gdavis331 (~gdavis@38.122.12.254) has joined #ceph
[21:58] <hasues> should the the osds in ceph be able to ssh to one another passwordless, or does the admin node simply need to ssh to each of them passwordless
[22:00] * jackhill (jackhill@pilot.trilug.org) Quit (Remote host closed the connection)
[22:02] <bens> monitors to storage nodes
[22:03] <bens> and managment to everything
[22:03] <bens> actually monitors to storage nodes I just use for convienence
[22:03] <bens> wherevery you run cephy deploy from to everything else, basically
[22:03] <hasues> bens: does the node that runs ceph-deploy need to be on the same subnet as those?
[22:06] <bens> nope.
[22:07] <bens> it is totally independent.
[22:07] * peetaur (~peter@x2f181f8.dyn.telefonica.de) Quit (Quit: Konversation terminated!)
[22:07] <bens> Sound like you are having a problem - theyc an't connect?
[22:07] * rotbeard (~redbeard@2a02:908:df19:7a80:76f0:6dff:fe3b:994d) Quit (Quit: Verlassend)
[22:07] <hasues> bens: I'm making a proof of concept ceph-cluster. I have two physical hosts and a vm, the vm is where I am running ceph-deploy, and it is on another subnet
[22:08] <hasues> I issued "ceph-deploy mon create-initial"
[22:09] <hasues> It bails because each monitor node I placed on the physical hosts states that "it is not yet in quorum", they time out, and it fails.
[22:09] <bens> did you do a new firet?
[22:09] <bens> *first
[22:09] <hasues> bens: yes I did a new <node1> <node2>
[22:10] <hasues> bens: install <ceph1> <ceph2>
[22:10] <bens> been a while since i did this, let me check something
[22:10] <hasues> bens: okay./
[22:11] <bens> does your ceph.conf look right?
[22:11] * Cube (~Cube@12.248.40.138) Quit (Quit: Leaving.)
[22:11] <hasues> bens: well, noting this one and one more will be the two I have ever seen. ;)
[22:12] <hasues> bens: it lists mon_hosts = <ip of node 1>,<ip of node 2>
[22:12] <bens> oh you need 3 monitors
[22:12] <bens> derp
[22:12] <hasues> mon_initial_members = <hostname1>, <hostname2>
[22:12] <hasues> oh?
[22:12] <bens> i just realized you only have 2.
[22:12] * Pedras (~Adium@64.191.206.83) has joined #ceph
[22:12] <bens> yeah, there has to be a majority to reach quorum.
[22:13] <bens> and 2 will tie.
[22:13] <hasues> bens: so I should put the third one on the vm then I suppose.
[22:14] <hasues> bens: can the admin node "ceph-deploy" to itself?
[22:15] * sputnik13 (~sputnik13@207.8.121.241) has joined #ceph
[22:15] * mattt_ (~textual@92.52.76.140) has joined #ceph
[22:16] <bens> yes
[22:17] <bens> i did all my deployments on my production cluster from mon #1
[22:17] <bens> including to mon # 1
[22:17] <hasues> bens: did you mon #1 have storage to participate?
[22:18] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[22:18] * mattt_ is now known as mattt
[22:20] * nwat (~textual@eduroam-246-164.ucsc.edu) has joined #ceph
[22:25] <hasues> bens: also, did you do a ceph-deploy install on mon1?
[22:25] * nwat (~textual@eduroam-246-164.ucsc.edu) Quit ()
[22:29] * allsystemsarego (~allsystem@188.25.129.255) Quit (Quit: Leaving)
[22:38] * mattt (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[22:40] * dmsimard (~Adium@108.163.152.2) Quit (Quit: Leaving.)
[22:40] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[22:40] * JoeGruher (~JoeGruher@jfdmzpr06-ext.jf.intel.com) has joined #ceph
[22:43] * linuxkidd (~linuxkidd@2001:420:2100:2258:39d3:de25:be2d:1e03) Quit (Quit: Leaving)
[22:49] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) Quit (Quit: Leaving)
[22:51] * Damage (~oftc-webi@81.95.228.112) has joined #ceph
[22:54] * srenatus (~stephan@e179112157.adsl.alicedsl.de) has joined #ceph
[22:55] <bens> hasues: sorry, i was afk. yes, i did ceph-deploy from mon1 to mon1. And I did this on both of my systems. One has OSDs on the mon, one does not. It doesn't matter.
[22:55] <hasues> bens: no worries, I appreciate what you have told me so far.
[22:56] <Damage> HI! Anybody speak russian? Can you advise me about using ceph?
[23:02] * srenatus (~stephan@e179112157.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[23:04] * mdjp (~mdjp@213.229.87.114) Quit (Ping timeout: 480 seconds)
[23:06] * mdjp (~mdjp@213.229.87.114) has joined #ceph
[23:08] * rendar (~s@host7-177-dynamic.20-87-r.retail.telecomitalia.it) Quit ()
[23:10] * BillK (~BillK-OFT@124-169-81-237.dyn.iinet.net.au) has joined #ceph
[23:17] * AfC (~andrew@2407:7800:400:1011:2ad2:44ff:fe08:a4c) has joined #ceph
[23:17] * kaizh (~kaizh@128-107-239-234.cisco.com) has joined #ceph
[23:18] * rudolfsteiner (~federicon@181.108.79.240) has joined #ceph
[23:19] * rudolfsteiner (~federicon@181.108.79.240) has left #ceph
[23:23] * markbby (~Adium@168.94.245.1) Quit (Quit: Leaving.)
[23:28] * yuriw (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[23:28] * yuriw (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) has joined #ceph
[23:31] <hasues> bens: This does not appear to work. I have exceptions being generated.
[23:32] <hasues> bens: If I want to wipe all of this and start over, the "ceph-deploy purgedata" and "forgetkeys" does not reset the environment as I see the nodes have ceph processes running.
[23:37] * JoeGruher (~JoeGruher@jfdmzpr06-ext.jf.intel.com) Quit (Remote host closed the connection)
[23:40] * Cube (~Cube@66-87-131-116.pools.spcsdns.net) has joined #ceph
[23:42] <svg> moh, krijg ne favorite van mijne tweet
[23:43] * BillK (~BillK-OFT@124-169-81-237.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[23:43] <bdonnahue2> im trying to create a btrfs osd
[23:43] <bdonnahue2> the ceph deploy is failing when preparing
[23:43] <bdonnahue2> i zapped the disks
[23:43] <bdonnahue2> then cat /proc/partitions showed no partitions on sdb
[23:44] <bdonnahue2> but i see
[23:44] <bdonnahue2> [WARNIN] ceph-disk: Error: Device is mounted: /dev/sdb1
[23:44] <bdonnahue2> [ERROR ] Failed to execute command: ceph-disk-prepare --fs-type btrfs --cluster ceph -- /dev/sdb
[23:44] <bdonnahue2> anyone know why?
[23:47] <bdonnahue2> ah i see this when i re run zap
[23:47] <bdonnahue2> [DEBUG ] Warning: The kernel is still using the old partition table.
[23:47] <bdonnahue2> do i need to reboot?
[23:49] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[23:51] * guppy (~quassel@guppy.xxx) Quit (Quit: No Ping reply in 180 seconds.)
[23:52] <dmick> well, uh, is /dev/sdb1 mounted?
[23:54] <bdonnahue2> im not sure. i zapped, now im rebooting.
[23:55] * Damage (~oftc-webi@81.95.228.112) Quit (Quit: Page closed)
[23:55] <bdonnahue2> im now seeing a new error
[23:55] <bdonnahue2> [WARNIN] mkfs.btrfs: No such file or directory
[23:55] <bdonnahue2> [WARNIN] ceph-disk: Error: Command '['mkfs', '-t', 'btrfs', '-m', 'single', '-l', '32768', '-n', '32768', '--', '/dev/sdb1']' returned non-zero exit status 1
[23:55] <bens> hasues: kill ceph.
[23:56] <bens> looks like btrfs isn't installed.
[23:58] <bdonnahue2> i figured ceph-deploy would install that on the osd?
[23:59] <bdonnahue2> im not sure what to do to install it

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.