#ceph IRC Log

Index

IRC Log for 2014-06-05

Timestamps are in GMT/BST.

[0:03] * Cube (~Cube@12.248.40.138) Quit (Ping timeout: 480 seconds)
[0:04] * sarob_ (~sarob@129.210.115.7) Quit (Remote host closed the connection)
[0:05] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) has joined #ceph
[0:11] * hasues (~hasues@kwfw01.scrippsnetworksinteractive.com) Quit (Quit: Leaving.)
[0:11] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) Quit (Read error: Connection reset by peer)
[0:12] * rweeks (~goodeats@c-24-6-118-113.hsd1.ca.comcast.net) has joined #ceph
[0:12] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) has joined #ceph
[0:13] * Infitialis (~infitiali@5ED48E69.cm-7-5c.dynamic.ziggo.nl) has joined #ceph
[0:14] * mjeanson (~mjeanson@00012705.user.oftc.net) Quit (Remote host closed the connection)
[0:15] * mjeanson (~mjeanson@bell.multivax.ca) has joined #ceph
[0:18] * cookednoodles (~eoin@eoin.clanslots.com) Quit (Quit: Ex-Chat)
[0:24] * primechuck (~primechuc@host-71-34-75.infobunker.com) Quit (Remote host closed the connection)
[0:24] <ikrstic> Finaly.... I see calamari dashboard :)
[0:25] <ircolle> ikrstic - congrats!
[0:30] * `jpg (~josephgla@ppp255-151.static.internode.on.net) has joined #ceph
[0:33] * lofejndif (~lsqavnbok@6FMAABDZN.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)
[0:35] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Quit: ...)
[0:37] * kfei (~root@114-27-53-229.dynamic.hinet.net) Quit (Read error: Operation timed out)
[0:41] * `jpg (~josephgla@ppp255-151.static.internode.on.net) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[0:42] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[0:43] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) Quit (Quit: Konversation terminated!)
[0:43] * gregmark (~Adium@cet-nat-254.ndceast.pa.bo.comcast.net) Quit (Quit: Leaving.)
[0:44] * Cube (~Cube@66.87.66.229) has joined #ceph
[0:51] * sarob (~sarob@129.210.115.7) has joined #ceph
[0:51] * Infitialis (~infitiali@5ED48E69.cm-7-5c.dynamic.ziggo.nl) Quit (Remote host closed the connection)
[0:54] * mjeanson (~mjeanson@00012705.user.oftc.net) Quit (Remote host closed the connection)
[0:54] <bens> yay for calamari being opensource
[0:54] <bens> now I can ask for help here! ;)
[0:55] * rturk is now known as rturk|afk
[0:55] * mjeanson (~mjeanson@bell.multivax.ca) has joined #ceph
[0:57] * Pauline_ (~middelink@bigbox.ch.polyware.nl) Quit (Ping timeout: 480 seconds)
[0:58] * thb (~me@0001bd58.user.oftc.net) Quit (Ping timeout: 480 seconds)
[0:58] * ssejour (~sebastien@ec135-1-78-239-10-19.fbx.proxad.net) Quit (Quit: Leaving.)
[0:58] * ssejour (~sebastien@ec135-1-78-239-10-19.fbx.proxad.net) has joined #ceph
[1:01] * `jpg (~josephgla@ppp255-151.static.internode.on.net) has joined #ceph
[1:02] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[1:04] * sarob_ (~sarob@129.210.115.7) has joined #ceph
[1:06] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[1:06] * ssejour (~sebastien@ec135-1-78-239-10-19.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[1:07] * bandrus1 (~Adium@66.87.118.32) Quit (Ping timeout: 480 seconds)
[1:09] * bandrus (~Adium@66.87.118.103) has joined #ceph
[1:10] * sarob (~sarob@129.210.115.7) Quit (Read error: Operation timed out)
[1:11] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) Quit (Quit: Leaving.)
[1:13] * gregsfortytwo (~Adium@129.210.115.6) Quit (Quit: Leaving.)
[1:16] * alop (~abelopez@128-107-239-235.cisco.com) Quit (Ping timeout: 480 seconds)
[1:21] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) has joined #ceph
[1:23] * sarob_ (~sarob@129.210.115.7) Quit (Remote host closed the connection)
[1:24] * jcsp1 (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Read error: Connection reset by peer)
[1:24] * sarob (~sarob@129.210.115.7) has joined #ceph
[1:24] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[1:24] * oms101 (~oms101@p20030057EA001A00EEF4BBFFFE0F7062.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[1:27] * gregsfortytwo (~Adium@129.210.115.6) has joined #ceph
[1:27] * ircolle (~Adium@mobile-198-228-212-145.mycingular.net) Quit (Quit: Leaving.)
[1:29] * jcsp (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[1:29] <seapasulli> anyone have ceph working as a cinder backend and have cinder error out after mass creation and deletion of 100 1TB volumes?
[1:29] * alop (~abelopez@128-107-239-234.cisco.com) has joined #ceph
[1:29] * Guest12144 (~coyo@thinks.outside.theb0x.org) Quit (Ping timeout: 480 seconds)
[1:29] <seapasulli> or really when trying to create then delete 100, it errors out at around 50
[1:30] <seapasulli> (44)
[1:30] <seapasulli> so rounded way up hahaha
[1:30] * gregsfortytwo (~Adium@129.210.115.6) Quit ()
[1:30] * gregsfortytwo (~Adium@129.210.115.6) has joined #ceph
[1:31] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[1:32] * sarob (~sarob@129.210.115.7) Quit (Ping timeout: 480 seconds)
[1:33] * oms101 (~oms101@p20030057EA342800EEF4BBFFFE0F7062.dip0.t-ipconnect.de) has joined #ceph
[1:34] * alop (~abelopez@128-107-239-234.cisco.com) Quit (Quit: alop)
[1:41] * Coyo (~coyo@thinks.outside.theb0x.org) has joined #ceph
[1:41] * Coyo is now known as Guest12597
[1:41] * AfC (~andrew@nat-gw2.syd4.anchor.net.au) has joined #ceph
[1:43] <seapasulli> now cinder thinks that the volumes still exist but rbd -p volumes ls shows around 27 left and cinder list shows 44. I'm deleting them manually from cinder
[1:43] <seapasulli> and ceph but don't know why it happened
[1:44] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) Quit (Quit: Leaving.)
[1:44] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) has left #ceph
[1:45] * Pauline (~middelink@bigbox.ch.polyware.nl) has joined #ceph
[1:46] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) Quit (Quit: Leaving.)
[1:49] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[1:51] * brytown (~Adium@2620:79:0:8204:6847:e1ba:65c4:a591) has joined #ceph
[1:52] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:53] * rweeks (~goodeats@c-24-6-118-113.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[1:54] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) has joined #ceph
[1:54] * gregsfortytwo (~Adium@129.210.115.6) Quit (Quit: Leaving.)
[1:55] * gregsfortytwo (~Adium@129.210.115.6) has joined #ceph
[1:55] * gregsfortytwo (~Adium@129.210.115.6) Quit ()
[1:56] * diegows (~diegows@190.190.5.238) Quit (Ping timeout: 480 seconds)
[2:06] * zerick (~eocrospom@190.187.21.53) Quit (Read error: Operation timed out)
[2:07] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) has joined #ceph
[2:10] * jcsp1 (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[2:10] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) has joined #ceph
[2:15] * jcsp (~Adium@0001bf3a.user.oftc.net) Quit (Ping timeout: 480 seconds)
[2:18] * bandrus (~Adium@66.87.118.103) Quit (Quit: Leaving.)
[2:22] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) Quit (Remote host closed the connection)
[2:22] * sjusthm (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) Quit (Quit: Leaving.)
[2:22] * rturk|afk is now known as rturk
[2:24] * ircolle (~Adium@156.39.127.195) has joined #ceph
[2:33] * bandrus (~Adium@66.87.118.103) has joined #ceph
[2:33] * bandrus1 (~Adium@66.87.118.103) has joined #ceph
[2:33] * bandrus (~Adium@66.87.118.103) Quit (Read error: Connection reset by peer)
[2:35] * KaZeR (~kazer@64.201.252.132) Quit (Read error: Operation timed out)
[2:35] * rmoe (~quassel@12.164.168.117) Quit (Ping timeout: 480 seconds)
[2:37] * xarses (~andreww@12.164.168.117) Quit (Ping timeout: 480 seconds)
[2:44] * brytown (~Adium@2620:79:0:8204:6847:e1ba:65c4:a591) Quit (Ping timeout: 480 seconds)
[2:47] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) has joined #ceph
[2:50] * LeaChim (~LeaChim@host86-174-77-240.range86-174.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:53] * KaZeR (~kazer@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[2:54] * bandrus1 (~Adium@66.87.118.103) Quit (Ping timeout: 480 seconds)
[2:56] * ircolle (~Adium@156.39.127.195) Quit (Quit: Leaving.)
[2:59] <classicsnail> okay, that's just so weird
[2:59] <classicsnail> ceph mds, the mds would never start, ultimately crashing after timeout
[2:59] <classicsnail> either standby or active, I had to go around, terminating all cephfs mounts on the network
[2:59] <classicsnail> do that, the mds comes up instantly
[3:00] * rturk is now known as rturk|afk
[3:02] * `jpg (~josephgla@ppp255-151.static.internode.on.net) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[3:05] * sputnik13 (~sputnik13@207.8.121.241) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[3:09] * narb (~Jeff@38.99.52.10) Quit (Quit: narb)
[3:10] * narb (~Jeff@38.99.52.10) has joined #ceph
[3:11] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) Quit (Read error: Operation timed out)
[3:16] * rturk|afk is now known as rturk
[3:23] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Read error: Operation timed out)
[3:25] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) has joined #ceph
[3:27] * angdraug (~angdraug@12.164.168.117) Quit (Quit: Leaving)
[3:30] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[3:34] * lucas1 (~Thunderbi@222.247.57.50) has joined #ceph
[3:36] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[3:37] * rturk is now known as rturk|afk
[3:45] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[3:49] * drankis_ (~drankis__@89.111.13.198) has joined #ceph
[4:01] * drankis_ (~drankis__@89.111.13.198) Quit (Ping timeout: 480 seconds)
[4:09] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[4:11] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[4:12] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[4:13] * `jpg (~josephgla@ppp255-151.static.internode.on.net) has joined #ceph
[4:19] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[4:21] * lucas1 (~Thunderbi@222.247.57.50) Quit (Ping timeout: 480 seconds)
[4:32] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[4:37] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[4:37] * zack_dol_ (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[4:37] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Read error: Connection reset by peer)
[4:57] * talonisx (~talonisx@pool-108-18-97-131.washdc.fios.verizon.net) Quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/)
[4:58] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[5:04] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) Quit (Quit: Leaving.)
[5:05] * ssejour (~sebastien@ec135-1-78-239-10-19.fbx.proxad.net) has joined #ceph
[5:06] <KB> anyone know if there's a minimum number of nodes, osds, or osd size for firefly? with 2 nodes, 3 osds, and 10gb per osd, the initial pools stay in active+degraded forever... I've set the osd pool default size to 2, which took effect, also tried setting the osd pool default pg num and pgp num to 8, which did NOT take effect...
[5:07] <KB> I'm guessing there's some limitation I'm hitting due to crushmap or # pgs, but can't figure out what...
[5:10] <dmick> no minimum number
[5:10] <lurbs> How are you checking if setting the default pg_num and pgp_num took effect?
[5:10] <dmick> "default" is "for new creation", don't affect existing pools
[5:10] <lurbs> Creating new pools, and checking how many they have?
[5:15] <KB> this is for creating the initial 3 pools - data, metadata, and rbd
[5:16] <KB> when I initiate the ceph-deploy mon create-initial, then add the 3 OSDs, the pools are created automatically with 64 pgs, and pgps
[5:16] <KB> even though the [global] section of my ceph.conf is set with "osd pool default pg num = 8" and "osd pool default pgp num = 8"
[5:17] <KB> the same section has "osd pool default size = 2" which is in effect - the ceph osd dump shows that size = 2, min_size = 1
[5:17] <KB> but pg_num = 64 and pgp_num = 64
[5:19] * Vacum (~vovo@88.130.193.115) has joined #ceph
[5:20] <lurbs> KB: Not sure why that's happening, but 64 probably is a more reasonable number of placement groups in any case.
[5:22] <KB> sure - makes sense to have the 64, just curious as to why these pools are all stuck in active+degraded. Even new pools with a small # of pgs/pgps are going into "creating", then a few minutes later, "active+degraded", with all 3 osd's up/in. odd.
[5:22] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[5:22] <KB> an emperor cluster that we upgraded from 0.72.2 to 0.80.1 has no issues - only fresh install of firefly.
[5:22] * yguang11 (~yguang11@2406:2000:ef96:e:8800:5a8e:8441:b8f8) has joined #ceph
[5:22] <dmick> is every pg degraded?
[5:23] <KB> yep
[5:23] <KB> 3 default pools x 64 pgs = 192 total - all active+degraded
[5:23] <dmick> does pg query on one of them give any insight?
[5:24] <KB> doesn't look like it to me, but: http://pastebin.com/aBSDc35J
[5:26] * Vacum_ (~vovo@88.130.216.40) Quit (Ping timeout: 480 seconds)
[5:26] <dmick> up/acting have only one OSD
[5:26] <dmick> that's a crush problem
[5:27] <KB> that's what it seemed like to me
[5:27] <dmick> likely choosing host first, then osd
[5:29] <KB> the ruleset is the default with chooseleaf firstn 0 type host, min_size 1
[5:30] <KB> both hosts and all 3 osds show in the crushmap too
[5:31] * rejy (~rmc@nat-pool-blr-t.redhat.com) has joined #ceph
[5:31] <KB> thought was with a pool size of 2, and having 2 hosts with an in/up osd, that should satisfy the crushmap
[5:35] <KB> ... and just updated the crushmap to type osd (which there are 3 of... all up/in), and no change.
[5:43] * jcsp (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[5:46] * jcsp1 (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[5:47] * reed (~reed@75-101-54-131.dsl.static.sonic.net) Quit (Quit: Ex-Chat)
[5:54] * zhaochao (~zhaochao@23.226.224.80) has joined #ceph
[5:54] <dmick> KB: can you post the crushtool -d of your map?
[5:58] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[5:58] * zack_dol_ (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Read error: No route to host)
[5:59] * Muhlemmer (~kvirc@cable-90-50.zeelandnet.nl) has joined #ceph
[6:03] * wschulze (~wschulze@cpe-69-206-251-158.nyc.res.rr.com) Quit (Quit: Leaving.)
[6:05] <KB> sure - sec
[6:06] <KB> http://pastebin.com/wL705bfc
[6:14] * saurabh (~saurabh@nat-pool-blr-t.redhat.com) has joined #ceph
[6:25] * Muhlemmer (~kvirc@cable-90-50.zeelandnet.nl) Quit (Ping timeout: 480 seconds)
[6:29] * ssejour (~sebastien@ec135-1-78-239-10-19.fbx.proxad.net) Quit (Quit: Leaving.)
[6:29] * sm1ly (~sm1ly@ppp109-252-169-241.pppoe.spdop.ru) has joined #ceph
[6:34] * saurabh (~saurabh@nat-pool-blr-t.redhat.com) Quit (Ping timeout: 480 seconds)
[6:42] * haomaiwang (~haomaiwan@112.193.130.58) has joined #ceph
[6:42] * haomaiwang (~haomaiwan@112.193.130.58) Quit (Remote host closed the connection)
[6:43] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[6:44] * gleam (gleam@dolph.debacle.org) Quit (Read error: Operation timed out)
[6:44] * gleam (gleam@dolph.debacle.org) has joined #ceph
[6:47] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Ping timeout: 480 seconds)
[6:47] * shalicke (~shalicke@192.241.186.125) Quit (Remote host closed the connection)
[6:48] * shalicke (~shalicke@192.241.186.125) has joined #ceph
[6:49] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[6:50] * AfC (~andrew@nat-gw2.syd4.anchor.net.au) Quit (Read error: Operation timed out)
[6:51] * saurabh (~saurabh@209.132.188.8) has joined #ceph
[6:57] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[7:00] * haomaiwa_ (~haomaiwan@112.193.130.58) has joined #ceph
[7:03] * lalatenduM (~lalatendu@nat-pool-blr-t.redhat.com) has joined #ceph
[7:03] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) Quit (Quit: Leaving.)
[7:03] * shang (~ShangWu@n119237225138.netvigator.com) has joined #ceph
[7:04] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Ping timeout: 480 seconds)
[7:09] * michalefty (~micha@p20030071CF6CAC004471EAC04B10452F.dip0.t-ipconnect.de) has joined #ceph
[7:09] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[7:12] * vbellur (~vijay@122.167.250.20) Quit (Quit: Leaving.)
[7:12] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) Quit (Remote host closed the connection)
[7:14] * jcsp1 (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[7:14] * haomaiwa_ (~haomaiwan@112.193.130.58) Quit (Remote host closed the connection)
[7:14] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[7:18] * jcsp (~Adium@0001bf3a.user.oftc.net) Quit (Ping timeout: 480 seconds)
[7:30] * haomaiwa_ (~haomaiwan@124.248.205.17) has joined #ceph
[7:33] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[7:35] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Ping timeout: 480 seconds)
[7:36] * sleinen (~Adium@84-72-160-233.dclient.hispeed.ch) has joined #ceph
[7:39] * sleinen1 (~Adium@2001:620:0:26:c1b9:cbda:6c7b:d4be) has joined #ceph
[7:39] * yguang11 (~yguang11@2406:2000:ef96:e:8800:5a8e:8441:b8f8) Quit (Remote host closed the connection)
[7:40] * yguang11 (~yguang11@2406:2000:ef96:e:8800:5a8e:8441:b8f8) has joined #ceph
[7:43] * vbellur (~vijay@209.132.188.8) has joined #ceph
[7:43] * sleinen (~Adium@84-72-160-233.dclient.hispeed.ch) Quit (Read error: Operation timed out)
[7:55] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[7:55] * sleinen1 (~Adium@2001:620:0:26:c1b9:cbda:6c7b:d4be) Quit (Quit: Leaving.)
[7:56] * haomaiwa_ (~haomaiwan@124.248.205.17) Quit (Ping timeout: 480 seconds)
[7:57] * haomaiwang (~haomaiwan@124.248.205.17) has joined #ceph
[7:57] * shang (~ShangWu@n119237225138.netvigator.com) Quit (Read error: Operation timed out)
[8:08] * madkiss (~madkiss@2001:6f8:12c3:f00f:1907:6e4a:d31a:7702) Quit (Ping timeout: 480 seconds)
[8:10] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[8:14] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) has joined #ceph
[8:15] * thb (~me@2a02:2028:210:fca0:6267:20ff:fec9:4e40) has joined #ceph
[8:16] * zack_dol_ (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[8:16] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Read error: Connection reset by peer)
[8:18] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[8:20] * danieljh_ (~daniel@HSI-KBW-046-005-197-128.hsi8.kabel-badenwuerttemberg.de) has joined #ceph
[8:22] * danieljh (~daniel@0001b4e9.user.oftc.net) Quit (Ping timeout: 480 seconds)
[8:24] * sleinen (~Adium@130.59.94.40) has joined #ceph
[8:26] * Sysadmin88 (~IceChat77@94.4.22.173) Quit (Quit: IceChat - Keeping PC's cool since 2000)
[8:26] * b0e (~aledermue@juniper1.netways.de) has joined #ceph
[8:26] * thomnico (~thomnico@2a01:e35:8b41:120:b841:4631:c163:3a27) has joined #ceph
[8:27] <classicsnail> is there a way to dump or remove the mdsmap?
[8:27] <classicsnail> say to "start again"?
[8:27] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[8:28] * ccourtaut (~ccourtaut@2001:41d0:2:4a25::1) Quit (Ping timeout: 480 seconds)
[8:28] * zack_dol_ (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Ping timeout: 480 seconds)
[8:29] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[8:30] * jcsp (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[8:31] * peedu (~peedu@mail.hadler.ee) has joined #ceph
[8:31] <peedu> hi
[8:32] <peedu> does anyone know how to ask ceph what is clusters nearfull ratio at the moment?
[8:32] <peedu> ceph pg set_nearfull_ratio i can set it with that
[8:32] * jcsp1 (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[8:32] <peedu> but how to know what is the ratio at the moment?
[8:39] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Remote host closed the connection)
[8:42] <peedu> ceph report | grep full_ratio
[8:42] <peedu> fast way to find
[8:47] <singler_> also you can check "ceph pg dump"
[8:48] * asfsa (~peng@119.2.3.164) has joined #ceph
[8:48] <asfsa> hi~
[8:49] <asfsa> anyone here ?
[8:49] <asfsa> I cann't add osd for my ceph cluster...
[8:49] * rendar (~I@host123-161-dynamic.1-87-r.retail.telecomitalia.it) has joined #ceph
[8:50] <asfsa> [10.41.3.12][INFO ] Running command: ceph-disk-activate --mark-init sysvinit --mount /root/osd4
[8:50] <asfsa> [10.41.3.12][WARNING] ceph-disk: Error: No cluster conf found in /etc/ceph with fsid 7e1c372f-ac9c-4b96-853c-c276b5cdce9d
[8:50] <asfsa> [10.41.3.12][ERROR ] RuntimeError: command returned non-zero exit status: 1
[8:50] <asfsa> [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph-disk-activate --mark-init sysvinit --mount /root/osd4
[8:50] <asfsa> ceph-disk-activate Fail !
[8:51] <asfsa> And the command "ceph -s" always tells me the old osds . I can't add more osd into the cluster.
[8:52] <singler_> is this a new host?
[8:52] * yuriw (~Adium@ABordeaux-654-1-80-223.w109-214.abo.wanadoo.fr) has joined #ceph
[8:52] <singler_> I think this is a problem "[WARNING] ceph-disk: Error: No cluster conf found in /etc/ceph with fsid 7e1c372f-ac9c-4b96-853c-c276b5cdce9d"
[8:53] <asfsa> en ,it 's a old host.
[8:53] * singler_ afk for a few minutes
[8:53] <asfsa> fsid in my ceph.conf is edc147e1-4db6-48fc-a6c6-6ea8941a97a4
[8:53] * ccourtaut (~ccourtaut@2001:41d0:2:4a25::1) has joined #ceph
[8:54] <asfsa> it is not 7e1c372f-ac9c-4b96-853c-c276b5cdce9d" .. It's wired.
[8:54] <asfsa> it is strange..
[8:55] <asfsa> I am trying to kill the old thread in my osd host.
[8:55] <asfsa> maybe it can solve the bug.
[8:56] * madkiss (~madkiss@212.17.93.220) has joined #ceph
[8:58] <asfsa> I have purged the old cluster. but it seems the old fsid is still active.
[8:59] <singler_> maybe ceph-deploy directory has old cluster's datad?
[9:04] * vbellur (~vijay@209.132.188.8) Quit (Ping timeout: 480 seconds)
[9:05] * oms101 (~oms101@p20030057EA342800EEF4BBFFFE0F7062.dip0.t-ipconnect.de) Quit (Quit: Leaving)
[9:05] * yguang11 (~yguang11@2406:2000:ef96:e:8800:5a8e:8441:b8f8) Quit (Remote host closed the connection)
[9:05] * mjeanson_ (~mjeanson@bell.multivax.ca) has joined #ceph
[9:06] * yguang11 (~yguang11@2406:2000:ef96:e:8800:5a8e:8441:b8f8) has joined #ceph
[9:08] * mjeanson (~mjeanson@00012705.user.oftc.net) Quit (Ping timeout: 480 seconds)
[9:12] * houkouonchi-home (~linux@2001:470:c:c69::2) Quit (Ping timeout: 480 seconds)
[9:14] * vbellur (~vijay@nat-pool-blr-t.redhat.com) has joined #ceph
[9:16] * Infitialis (~infitiali@194.30.182.18) has joined #ceph
[9:18] * analbeard (~shw@support.memset.com) has joined #ceph
[9:19] * houkouonchi-home (~linux@pool-71-189-160-82.lsanca.fios.verizon.net) has joined #ceph
[9:22] * ajazdzewski (~quassel@lpz-66.sprd.net) has joined #ceph
[9:24] * thomnico (~thomnico@2a01:e35:8b41:120:b841:4631:c163:3a27) Quit (Quit: Ex-Chat)
[9:27] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) has joined #ceph
[9:29] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[9:35] * Nacer (~Nacer@252-87-190-213.intermediasud.com) has joined #ceph
[9:35] * peedu (~peedu@mail.hadler.ee) Quit (Read error: Connection reset by peer)
[9:36] * peedu (~peedu@mail.hadler.ee) has joined #ceph
[9:36] * mjeanson_ is now known as mjeanson
[9:38] * zack_dol_ (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[9:38] * yuriw (~Adium@ABordeaux-654-1-80-223.w109-214.abo.wanadoo.fr) Quit (Quit: Leaving.)
[9:40] * odyssey4me (~odyssey4m@165.233.71.2) has joined #ceph
[9:41] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Ping timeout: 480 seconds)
[9:41] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[9:41] * ade (~abradshaw@193.202.255.218) has joined #ceph
[9:42] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit ()
[9:42] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[9:44] * TMM (~hp@178-84-46-106.dynamic.upc.nl) Quit (Ping timeout: 480 seconds)
[9:45] * yguang11 (~yguang11@2406:2000:ef96:e:8800:5a8e:8441:b8f8) Quit (Remote host closed the connection)
[9:46] * zack_dol_ (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Ping timeout: 480 seconds)
[9:46] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) has joined #ceph
[9:46] * ScOut3R (~ScOut3R@catv-89-133-22-210.catv.broadband.hu) has joined #ceph
[9:52] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) has joined #ceph
[9:53] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) Quit (Read error: Operation timed out)
[10:00] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) Quit (Quit: Leaving.)
[10:01] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) has joined #ceph
[10:02] * saurabh (~saurabh@209.132.188.8) Quit (Ping timeout: 480 seconds)
[10:03] * haomaiwang (~haomaiwan@124.248.205.17) Quit (Ping timeout: 480 seconds)
[10:03] * cookednoodles (~eoin@eoin.clanslots.com) has joined #ceph
[10:05] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[10:07] * haomaiwang (~haomaiwan@112.193.130.58) has joined #ceph
[10:09] * shang (~ShangWu@42-64-93-170.dynamic-ip.hinet.net) has joined #ceph
[10:10] * saurabh (~saurabh@nat-pool-blr-t.redhat.com) has joined #ceph
[10:18] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) Quit (Quit: Konversation terminated!)
[10:19] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[10:21] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[10:21] * zidarsk8 (~zidar@prevod.fri1.uni-lj.si) has joined #ceph
[10:22] * q53 (~nospam10@94.19.218.213) has joined #ceph
[10:22] * q53 (~nospam10@94.19.218.213) has left #ceph
[10:22] * q53 (~nospam10@94.19.218.213) has joined #ceph
[10:23] <q53> test
[10:23] <absynth> test worked
[10:23] * oms101 (~oms101@p20030057EA342800EEF4BBFFFE0F7062.dip0.t-ipconnect.de) has joined #ceph
[10:24] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[10:26] <q53> Hi, all
[10:26] <q53> is there a way quickly fix/remove incomplete PG?
[10:26] <q53> ceph 0.72.2 on SL6.5
[10:26] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[10:27] <q53> pg force_create_pg does not work
[10:27] * oms101 (~oms101@p20030057EA342800EEF4BBFFFE0F7062.dip0.t-ipconnect.de) Quit (Remote host closed the connection)
[10:28] * LeaChim (~LeaChim@host86-174-77-240.range86-174.btcentralplus.com) has joined #ceph
[10:33] * hijacker (~hijacker@213.91.163.5) Quit (Quit: Leaving)
[10:34] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[10:34] * haomaiwang (~haomaiwan@112.193.130.58) Quit (Remote host closed the connection)
[10:35] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[10:35] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) Quit (Quit: Leaving.)
[10:36] * lucas1 (~Thunderbi@218.76.25.66) has joined #ceph
[10:40] * ksperis (~ksperis@46.218.42.103) has joined #ceph
[10:40] * allsystemsarego (~allsystem@188.27.188.69) has joined #ceph
[10:41] * haomaiwa_ (~haomaiwan@112.193.130.58) has joined #ceph
[10:42] * haomaiwa_ (~haomaiwan@112.193.130.58) Quit (Remote host closed the connection)
[10:42] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Read error: Connection reset by peer)
[10:43] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[10:43] * ikrstic (~ikrstic@c82-214-88-26.loc.akton.net) has joined #ceph
[10:43] <q53> nobody knows?
[10:45] * m0e (~Moe@41.45.208.72) has joined #ceph
[10:47] * oms101 (~oms101@p20030057EA342800EEF4BBFFFE0F7062.dip0.t-ipconnect.de) has joined #ceph
[10:48] * Cube (~Cube@66.87.66.229) Quit (Quit: Leaving.)
[10:50] * haomaiwa_ (~haomaiwan@112.193.130.58) has joined #ceph
[10:52] * haomaiwa_ (~haomaiwan@112.193.130.58) Quit (Remote host closed the connection)
[10:52] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Read error: Connection reset by peer)
[10:52] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[10:56] * blue (~blue@irc.mmh.dk) has left #ceph
[10:59] * aldavud (~aldavud@213.55.176.180) has joined #ceph
[11:04] * haomaiwa_ (~haomaiwan@112.193.130.58) has joined #ceph
[11:05] * vbellur (~vijay@nat-pool-blr-t.redhat.com) Quit (Ping timeout: 480 seconds)
[11:08] <zidarsk8> q53: sorry, I can't help you there
[11:08] * zidarsk8 (~zidar@prevod.fri1.uni-lj.si) has left #ceph
[11:11] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Ping timeout: 480 seconds)
[11:12] * eternaleye (~eternaley@50.245.141.73) Quit (Ping timeout: 480 seconds)
[11:13] * eternaleye (~eternaley@50.245.141.73) has joined #ceph
[11:14] * aldavud (~aldavud@213.55.176.180) Quit (Remote host closed the connection)
[11:14] * aldavud (~aldavud@213.55.176.180) has joined #ceph
[11:18] * jack (~jack@178.250.208.80) Quit (Quit: leaving)
[11:18] * shang (~ShangWu@42-64-93-170.dynamic-ip.hinet.net) Quit (Read error: Connection reset by peer)
[11:21] * eternaleye (~eternaley@50.245.141.73) Quit (Read error: Operation timed out)
[11:21] * vbellur (~vijay@209.132.188.8) has joined #ceph
[11:23] * Ponyo (~fuzzy@c-98-232-38-159.hsd1.wa.comcast.net) has joined #ceph
[11:25] * lucas1 (~Thunderbi@218.76.25.66) Quit (Quit: lucas1)
[11:26] * jack (~jack@178.250.208.80) has joined #ceph
[11:26] * jack (~jack@178.250.208.80) Quit ()
[11:26] * jack (~jack@178.250.208.80) has joined #ceph
[11:30] * jack (~jack@178.250.208.80) Quit ()
[11:30] * jack (~jack@178.250.208.80) has joined #ceph
[11:33] * m0e (~Moe@41.45.208.72) Quit (Quit: This computer has gone to sleep)
[11:33] * eternaleye (~eternaley@50.245.141.73) has joined #ceph
[11:33] * `jpg (~josephgla@ppp255-151.static.internode.on.net) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[11:35] * jack (~jack@178.250.208.80) Quit ()
[11:35] * jack (~jack@178.250.208.80) has joined #ceph
[11:37] * jack (~jack@178.250.208.80) Quit ()
[11:37] * jack (~jack@178.250.208.80) has joined #ceph
[11:37] * zack_dolby (~textual@e0109-114-22-4-235.uqwimax.jp) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[11:38] * hybrid512 (~walid@195.200.167.70) has joined #ceph
[11:39] * hybrid512 (~walid@195.200.167.70) Quit ()
[11:39] * hybrid512 (~walid@195.200.167.70) has joined #ceph
[11:40] * lucas1 (~Thunderbi@218.76.25.66) has joined #ceph
[11:41] * lucas1 (~Thunderbi@218.76.25.66) Quit ()
[11:45] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[11:46] * Cube (~Cube@66-87-66-229.pools.spcsdns.net) has joined #ceph
[11:49] * jeremy__s (~jeremy@LReunion-151-3-51.w193-253.abo.wanadoo.fr) has joined #ceph
[11:49] <jeremy__s> Hello everyone, i have a question about ceph : what are the risks of creating a cluster with different OSD size, e.g : 500Gb/136Gb/1Tb.. performance issue ? anything else?
[11:53] <tnt> You'll have to watch the datadistribution more closely. The ones with a higher capacity will also have more load than the others.
[11:55] * lucas1 (~Thunderbi@222.240.148.154) has joined #ceph
[11:59] * Cube (~Cube@66-87-66-229.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[12:00] <jeremy__s> tnt: ok, why more load on the bigger OSD? does this means that i'll have to manipulate crush maps in order to make things working correctly or just watching about the ratio ?
[12:00] <Serbitar> more load because it will store more objects
[12:01] <Ponyo> Hi does anyone know if any documented use cases of using Ceph to hold user home folders?
[12:02] * lucas1 (~Thunderbi@222.240.148.154) Quit (Quit: lucas1)
[12:04] * aldavud (~aldavud@213.55.176.180) Quit (Ping timeout: 480 seconds)
[12:04] * ScOut3R (~ScOut3R@catv-89-133-22-210.catv.broadband.hu) Quit (Read error: Operation timed out)
[12:11] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[12:14] <jeremy__s> okay, what about replication, the fact of having multiple OSD size will make the process working differently ?
[12:17] * ScOut3R (~ScOut3R@catv-80-99-64-8.catv.broadband.hu) has joined #ceph
[12:18] <tnt> jeremy__s: yes, it will put more "PGs" on the bigger OSD.
[12:24] * zack_dolby (~textual@e0109-114-22-12-137.uqwimax.jp) has joined #ceph
[12:27] * thorus (~jonas@158.181.55.4) has left #ceph
[12:32] * jack (~jack@178.250.208.80) Quit (Quit: leaving)
[12:32] * jack (~jack@178.250.208.80) has joined #ceph
[12:32] * jack (~jack@178.250.208.80) Quit ()
[12:32] * jack (~jack@178.250.208.80) has joined #ceph
[12:35] * jack (~jack@178.250.208.80) Quit ()
[12:35] * jack (~jack@178.250.208.80) has joined #ceph
[12:36] * jack (~jack@178.250.208.80) Quit ()
[12:36] * jack (~jack@178.250.208.80) has joined #ceph
[12:37] * shang (~ShangWu@42-64-93-170.dynamic-ip.hinet.net) has joined #ceph
[12:39] * asfsa (~peng@119.2.3.164) Quit (Ping timeout: 480 seconds)
[12:40] * haomaiwa_ (~haomaiwan@112.193.130.58) Quit (Remote host closed the connection)
[12:40] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[12:41] * i_m (~ivan.miro@gbibp9ph1--blueice1n2.emea.ibm.com) has joined #ceph
[12:41] * i_m (~ivan.miro@gbibp9ph1--blueice1n2.emea.ibm.com) Quit ()
[12:42] * i_m (~ivan.miro@gbibp9ph1--blueice4n1.emea.ibm.com) has joined #ceph
[12:44] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) Quit (Ping timeout: 480 seconds)
[12:47] * haomaiwa_ (~haomaiwan@112.193.130.58) has joined #ceph
[12:49] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) has joined #ceph
[12:54] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Ping timeout: 480 seconds)
[12:57] <jeremy__s> tnt: thanks
[13:00] * q53 (~nospam10@94.19.218.213) has left #ceph
[13:01] * sleinen (~Adium@130.59.94.40) Quit (Quit: Leaving.)
[13:03] <tnt> What's the easiest way to trace _where_ IOs are coming from / to ?
[13:04] <liiwi> iotop
[13:04] <Ponyo> love iotop
[13:05] <tnt> I meant on the ceph cluster ... I have IO/s hitting the OSD and I'd like to trace it to a client.
[13:05] <Ponyo> nettop maybe?
[13:08] <canta> iftop
[13:08] <Ponyo> that's it, I knew there was something better than ntop
[13:09] * zack_dolby (~textual@e0109-114-22-12-137.uqwimax.jp) Quit (Read error: Connection reset by peer)
[13:09] <darkfader> if you want that constantly tracked you could look into netflow/ipfix senders for linux
[13:10] * zack_dolby (~textual@e0109-114-22-12-137.uqwimax.jp) has joined #ceph
[13:10] <darkfader> and then just filter to incoming connections to the osd ports from outside the ceph cluster
[13:11] <darkfader> (don't run it in a vm on the same ceph cluster though or it'll slightly fudge the results)
[13:11] * nljmo (~nljmo@5ED6C263.cm-7-7d.dynamic.ziggo.nl) Quit (Quit: Textual IRC Client: www.textualapp.com)
[13:13] <tnt> Ok, I traced it down to one of the Dom0 running RBD. Is there a way to associate a tcp port with a rbd image ?
[13:14] <darkfader> if you run iotop there now or use xentop (sort by vbd) you should see the VM that is causing it
[13:15] <tnt> Unfortunately the xm top shows 'total' sector written /read and not the current rate.
[13:16] <tnt> But I think I'll just run iotop on the various VMs there is not that many.
[13:16] <tnt> I wish RBD had some built-in accounting :p
[13:17] <darkfader> i still don't understand. if you do iotop on the dom0 you should see a number of blkback processes
[13:17] <darkfader> and one of them being the performance eater
[13:18] <darkfader> their naming is blkback.domid.vbdnum
[13:18] <darkfader> so then xl list | grep domid -> the one to hit
[13:18] <tnt> oh sorry, I only tried xm top, didn't try iotop there.
[13:19] * rdas (~rdas@122.168.212.224) has joined #ceph
[13:19] * sleinen (~Adium@user-28-19.vpn.switch.ch) has joined #ceph
[13:19] <darkfader> and ++ for accouting and QoS as wishlist items :)
[13:20] <darkfader> but you'd need to boot the vms off rbd natively
[13:20] <darkfader> otherwise you'll always just see the vm host
[13:22] * zack_dolby (~textual@e0109-114-22-12-137.uqwimax.jp) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[13:23] * sleinen (~Adium@user-28-19.vpn.switch.ch) Quit ()
[13:25] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) has joined #ceph
[13:25] <tnt> darkfader: well, I'd like per-image accounting :)
[13:25] <tnt> from the image name, I know who it is :p
[13:26] * sleinen (~Adium@user-28-13.vpn.switch.ch) has joined #ceph
[13:31] * wschulze (~wschulze@cpe-69-206-251-158.nyc.res.rr.com) has joined #ceph
[13:31] * shang (~ShangWu@42-64-93-170.dynamic-ip.hinet.net) Quit (Read error: Connection reset by peer)
[13:35] * Cube (~Cube@66-87-66-229.pools.spcsdns.net) has joined #ceph
[13:35] * sleinen (~Adium@user-28-13.vpn.switch.ch) Quit (Read error: Connection reset by peer)
[13:37] * gford (~fford@p5099fb23.dip0.t-ipconnect.de) has joined #ceph
[13:39] * saurabh (~saurabh@nat-pool-blr-t.redhat.com) Quit (Quit: Leaving)
[13:40] <darkfader> tnt: ok right, that makes it a little easier
[13:41] * keds (Ked@cpc6-pool14-2-0-cust202.15-1.cable.virginm.net) Quit ()
[13:43] * Cube (~Cube@66-87-66-229.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[13:45] * peedu (~peedu@mail.hadler.ee) Quit (Quit: Leaving...)
[13:50] * jcsp (~Adium@0001bf3a.user.oftc.net) Quit (Quit: Leaving.)
[13:51] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) has joined #ceph
[13:55] * vbellur (~vijay@209.132.188.8) Quit (Quit: Leaving.)
[14:00] * Infitialis (~infitiali@194.30.182.18) Quit ()
[14:02] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[14:03] * ganders (~root@200-127-158-54.net.prima.net.ar) has joined #ceph
[14:03] * sleinen (~Adium@194.230.155.137) has joined #ceph
[14:05] * rwheeler (~rwheeler@173.48.207.57) Quit (Quit: Leaving)
[14:09] * hijacker (~hijacker@bgva.sonic.taxback.ess.ie) has joined #ceph
[14:12] * sleinen (~Adium@194.230.155.137) Quit (Ping timeout: 480 seconds)
[14:15] * ccourtaut_ (~ccourtaut@2001:41d0:1:eed3::1) Quit (Quit: Leaving)
[14:18] * rdas (~rdas@122.168.212.224) Quit (Quit: Leaving)
[14:19] * ccourtaut_ (~ccourtaut@2001:41d0:1:eed3::1) has joined #ceph
[14:23] * jtangwk (~Adium@gateway.tchpc.tcd.ie) Quit (Quit: Leaving.)
[14:24] * markbby (~Adium@168.94.245.2) has joined #ceph
[14:25] * thomnico (~thomnico@2a01:e35:8b41:120:b841:4631:c163:3a27) has joined #ceph
[14:35] * zhaochao (~zhaochao@23.226.224.80) has left #ceph
[14:35] <jeremy__s> is it feasible to create an OSD over an iscsi disk attached to a ceph node (2x1Gb LACP network) ?
[14:37] * michalefty (~micha@p20030071CF6CAC004471EAC04B10452F.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[14:47] <dmsimard> jeremy__s: In theory, all ceph cares about is a block device. It could be a USB key if you wanted to.
[14:48] * `jpg (~josephgla@ppp255-151.static.internode.on.net) has joined #ceph
[14:48] <dmsimard> Now, wether or not that will perform as well as you'd hope it would be, I don't know.
[14:48] * michalefty (~micha@p20030071CF63F8004471EAC04B10452F.dip0.t-ipconnect.de) has joined #ceph
[14:48] <dmsimard> If you're exporting that iSCSI device from a SAN or a redundant storage array, you're starting to have a lot of overhead
[14:49] <dmsimard> 2 Gbps shouldn't be too much of a bottleneck for a single drive (~250Mb second) but you probably don't want to have too many drives using that link
[14:51] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) Quit (Quit: Leaving.)
[14:51] * `jpg (~josephgla@ppp255-151.static.internode.on.net) Quit ()
[14:51] * andreask (~andreask@zid-vpnn077.uibk.ac.at) has joined #ceph
[14:51] * ChanServ sets mode +v andreask
[14:52] <jeremy__s> dmsimard: indeed, the idea behind this is to export iSCSI devices from two NAS (3 disk each) which is connected to that dedicated 2Gbps network, total number of disks : 6
[14:52] <jeremy__s> and then build OSD on top of them
[14:53] <dmsimard> jeremy__s: Is the NAS really relevant if you plan on using Ceph ? Perhaps the NAS has also other uses ?
[14:54] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[14:55] * vbellur (~vijay@122.172.244.34) has joined #ceph
[14:55] <jeremy__s> dmsimard: it will be dedicated to this function, i'm trying to find the best architecture with hardware that i have. the company that i work for has totally freezed investment for their IT until the next year, so.. =)
[14:56] * andreask (~andreask@zid-vpnn077.uibk.ac.at) has left #ceph
[14:56] <dmsimard> Ah, yeah.. budget.
[14:56] <singler_> jeremy__s: just take out the NAS disks and put them in servers.. :)
[14:57] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) has joined #ceph
[14:57] * jtangwk (~Adium@gateway.tchpc.tcd.ie) has joined #ceph
[14:57] <jeremy__s> singler_: I have 3 servers with 2.5inches disk controllers and disk in the NAS are 3.5inches, huho =)
[14:57] <jeremy__s> s/controllers/bay/g
[14:57] <kraken> jeremy__s meant to say: singler_: I have 3 servers with 2.5inches disk bay/g and disk in the NAS are 3.5inches, huho =)
[14:58] <jeremy__s> kraken: thanks :)
[14:58] <alfredodeza> hrmn
[14:58] * alfredodeza thinks kraken should answer to that thanks
[14:59] <singler_> jeremy__s: nothing what a saw and some tape wouldn't fix... :)
[14:59] * michalefty (~micha@p20030071CF63F8004471EAC04B10452F.dip0.t-ipconnect.de) has left #ceph
[15:00] <jeremy__s> lol =)
[15:00] <alfredodeza> thanks kraken
[15:00] * kraken is astonished by the equipotent affirmation of thankfulness
[15:00] * rwheeler (~rwheeler@nat-pool-bos-t.redhat.com) has joined #ceph
[15:01] * sleinen1 (~Adium@2001:620:0:26:b96a:b144:5e23:12a1) has joined #ceph
[15:03] * DV__ (~veillard@libvirt.org) Quit (Remote host closed the connection)
[15:03] * DV (~veillard@libvirt.org) Quit (Remote host closed the connection)
[15:03] * DV (~veillard@2001:41d0:1:d478::1) has joined #ceph
[15:03] * DV__ (~veillard@libvirt.org) has joined #ceph
[15:04] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[15:04] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[15:07] * vbellur (~vijay@122.172.244.34) Quit (Ping timeout: 480 seconds)
[15:07] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[15:08] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[15:09] * scuttlemonkey (~scuttlemo@c-107-5-193-244.hsd1.mi.comcast.net) Quit (Ping timeout: 480 seconds)
[15:09] * sleinen1 (~Adium@2001:620:0:26:b96a:b144:5e23:12a1) Quit (Ping timeout: 480 seconds)
[15:10] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[15:10] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[15:11] * kwaegema (~kwaegema@daenerys.ugent.be) Quit (Ping timeout: 480 seconds)
[15:12] * jeremy__s (~jeremy@LReunion-151-3-51.w193-253.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[15:13] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[15:13] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[15:14] <alfredodeza> kraken: thanks
[15:14] <kraken> alfredodeza: no problem
[15:14] <alfredodeza> \o/
[15:17] * sroy (~sroy@2607:fad8:4:6:3e97:eff:feb5:1e2b) has joined #ceph
[15:17] * markbby (~Adium@168.94.245.2) Quit (Quit: Leaving.)
[15:19] * kwaegema (~kwaegema@gast073l.ugent.be) has joined #ceph
[15:21] * ivan` (~ivan`@000130ca.user.oftc.net) Quit (Quit: ERC Version 5.3 (IRC client for Emacs))
[15:26] * The_Bishop_ (~bishop@f055213012.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[15:28] * ivan` (~ivan`@000130ca.user.oftc.net) has joined #ceph
[15:30] * gregmark (~Adium@68.87.42.115) has joined #ceph
[15:32] <dmsimard> thank you kraken
[15:32] * kraken is engulfed by the strong asseveration of esteem
[15:32] <dmsimard> kraken: thanks
[15:32] * kraken is engulfed by the strong declaration of admiration
[15:32] <dmsimard> oh, sophisticated too.
[15:33] * vbellur (~vijay@122.167.205.178) has joined #ceph
[15:33] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[15:34] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) Quit (Quit: Leaving)
[15:40] * `jpg (~josephgla@ppp255-151.static.internode.on.net) has joined #ceph
[15:43] * kfei (~root@61-227-52-61.dynamic.hinet.net) has joined #ceph
[15:43] * shang (~ShangWu@ipvpn110138.netvigator.com) has joined #ceph
[15:48] * The_Bishop (~bishop@f055213012.adsl.alicedsl.de) has joined #ceph
[15:50] * ghartz (~ghartz@ircad17.u-strasbg.fr) Quit (Quit: Quitte)
[15:50] * madkiss (~madkiss@212.17.93.220) Quit (Ping timeout: 480 seconds)
[15:57] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) has joined #ceph
[16:03] * stewiem2000 (~stewiem20@195.10.250.233) Quit (Quit: Leaving.)
[16:05] * kwaegema (~kwaegema@gast073l.ugent.be) Quit (Ping timeout: 480 seconds)
[16:06] * stewiem2000 (~stewiem20@195.10.250.233) has joined #ceph
[16:12] * `jpg (~josephgla@ppp255-151.static.internode.on.net) Quit (Ping timeout: 480 seconds)
[16:12] * stewiem2000 (~stewiem20@195.10.250.233) Quit (Quit: Leaving.)
[16:15] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[16:16] * rpowell (~rpowell@128.135.219.215) has joined #ceph
[16:17] * kwaegema (~kwaegema@daenerys.ugent.be) has joined #ceph
[16:19] * stewiem2000 (~stewiem20@195.10.250.233) has joined #ceph
[16:19] * dignus (~jkooijman@t-x.dignus.nl) Quit (Ping timeout: 480 seconds)
[16:19] * diegows (~diegows@190.190.5.238) has joined #ceph
[16:20] * fghaas (~florian@212095007100.public.telering.at) has joined #ceph
[16:20] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) has joined #ceph
[16:20] * i_m (~ivan.miro@gbibp9ph1--blueice4n1.emea.ibm.com) Quit (Quit: Leaving.)
[16:21] * haomaiwa_ (~haomaiwan@112.193.130.58) Quit (Ping timeout: 480 seconds)
[16:22] * haomaiwang (~haomaiwan@112.193.130.58) has joined #ceph
[16:22] * markbby (~Adium@168.94.245.1) has joined #ceph
[16:23] * ikrstic (~ikrstic@c82-214-88-26.loc.akton.net) Quit (Quit: Konversation terminated!)
[16:28] * haomaiwang (~haomaiwan@112.193.130.58) Quit (Read error: Connection reset by peer)
[16:30] <ibuclaw> Hi, I'm trying to enamble usage logging, but it doesn't seem to logging anything
[16:30] <ibuclaw> Just returns: { "entries": [], "summary": []}
[16:31] <ibuclaw> I have [client.radosgw.gateway]
[16:31] <ibuclaw> rgw enable usage log = true
[16:32] <ibuclaw> Though I do note that the .usage pool does not exist
[16:32] <ibuclaw> should I create it manually?
[16:33] * fghaas (~florian@212095007100.public.telering.at) has left #ceph
[16:36] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) Quit (Ping timeout: 480 seconds)
[16:36] * zerick (~eocrospom@190.118.43.113) has joined #ceph
[16:40] * haomaiwa_ (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[16:41] * ghartz (~ghartz@ircad17.u-strasbg.fr) has joined #ceph
[16:41] * bandrus (~Adium@adsl-75-5-249-229.dsl.scrm01.sbcglobal.net) has joined #ceph
[16:43] * madkiss (~madkiss@2001:6f8:12c3:f00f:d479:dc1d:69fc:41dd) has joined #ceph
[16:46] * blinky_ghost (~psousa@213.228.167.67) has joined #ceph
[16:46] <blinky_ghost> hi all, I have my openstack cluster running integrated with ceph with 4 mon and 2 osds. Doing some dd disk performance tests inside my instance I get 30MB/s writing. Any hints to improve performance? Thanks
[16:48] * haomaiwa_ (~haomaiwan@li634-52.members.linode.com) Quit (Ping timeout: 480 seconds)
[16:49] * haomaiwang (~haomaiwan@112.193.130.58) has joined #ceph
[16:49] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) has joined #ceph
[16:52] * KaZeR (~kazer@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[16:52] * rpowell (~rpowell@128.135.219.215) has left #ceph
[16:55] * haomaiwa_ (~haomaiwan@124.161.72.250) has joined #ceph
[16:55] <ghartz> blinky_ghost, decrease your "size" pool (number of replica)
[16:55] <ghartz> use "rados bench" instead of dd
[16:55] * odyssey4me_ (~odyssey4m@165.233.71.2) has joined #ceph
[16:56] * KaZeR (~kazer@64.201.252.132) has joined #ceph
[16:56] <blinky_ghost> ghartz: I have osd_pool_default_size = 2, osd_journal_size = 2048, osd_pool_default_min_size = 1, osd_pool_default_pg_num = 100, osd_mkfs_type = xfs
[16:57] * jcsp (~Adium@0001bf3a.user.oftc.net) has joined #ceph
[16:57] <ghartz> blinky_ghost, I have the same result with dd and size 2
[16:57] * haomaiwang (~haomaiwan@112.193.130.58) Quit (Ping timeout: 480 seconds)
[16:57] <ghartz> if you use rados bench, you will have 60MB/s
[16:57] <blinky_ghost> ghartz: I have to install ceph inside my vm? to run rados bench?
[16:58] <ghartz> yep
[16:58] * primechuck (~primechuc@173-17-128-36.client.mchsi.com) has joined #ceph
[16:58] <ghartz> and have all the conf/keyring inside /etc/ceph
[17:00] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[17:00] <blinky_ghost> ghartz: ok thanks :) another question :) I rebooted one of my OSD servers to test HA. Everything is working fine after the reboot but now I see this in the logs: 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-1) set_extsize: FSSETXATTR: (22) Invalid argument
[17:01] <blinky_ghost> Is this a bug or something?
[17:01] <ghartz> looks like a bug
[17:01] <ghartz> never had it before
[17:01] <ghartz> HA works great
[17:02] * b0e (~aledermue@juniper1.netways.de) Quit (Ping timeout: 480 seconds)
[17:02] * cronix (~cronix@5.199.139.166) has joined #ceph
[17:02] * odyssey4me (~odyssey4m@165.233.71.2) Quit (Ping timeout: 480 seconds)
[17:03] <cronix> hi
[17:03] <cronix> i have a issue with an ceph cluster currently
[17:03] <blinky_ghost> ghartz: Yes it seems so :) I cold rebooted my 2 OSD servers at the same time and after reboot everything started working, I just had to restart ceph daemons.
[17:03] <cronix> currently there are 50% of my PG's in the peering state
[17:03] <cronix> and about 400 OSD's are down
[17:04] <cronix> the cluster state is changing rapidly getting better and worse every few seconds
[17:04] <ghartz> blinky_ghost, you should not have your OSD servers rebooting at the same time
[17:04] <cronix> sage: you're here?
[17:05] <absynth> how many OSDs do you have in total, cronix?
[17:06] <cronix> 1440
[17:06] <cronix> atm were @ 585 in down state
[17:06] <cronix> http://pastebin.com/za5EkxTe
[17:06] <cronix> verbose cluster state atm
[17:06] <absynth> is the osdmap "flickering", i.e. OSDs going down and up rapidly?
[17:06] * ganders (~root@200-127-158-54.net.prima.net.ar) Quit (Quit: WeeChat 0.4.1)
[17:06] <cronix> jep
[17:07] <cronix> thats exactly whats happening
[17:07] * ganders (~root@200-127-158-54.net.prima.net.ar) has joined #ceph
[17:07] <cronix> also the logs say that they cant ping each other
[17:07] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[17:07] <absynth> well, let the disclaimer come first: I'm just a user, not an employee, and what I say might be complete and utter bullshit
[17:07] <absynth> that said, did you try setting noout to have the cluster "cool down" itself to a stable state?
[17:08] <cronix> disclaimer read and acknowledged
[17:08] <cronix> not yet
[17:08] <absynth> you seem to not have unfounds yet, so basically all data is still available
[17:08] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[17:08] <cronix> were in evaluation
[17:08] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[17:08] <cronix> were more curious about what the hell happened to cause this behaviour
[17:08] <cronix> no real usage data is in the cluster so far
[17:09] <absynth> network issue? do you have cacti or something on the network
[17:09] <cronix> no network issue according to out network dept
[17:09] <absynth> mh
[17:10] <absynth> anything interesting in ceph -w? "wrongly marked me down" messages or stuff?
[17:10] <absynth> clock skew detected?
[17:10] * alfredodeza (~alfredode@198.206.133.89) has left #ceph
[17:10] <cronix> 0.80.1 -> 0.72.2
[17:10] <cronix> version screwup
[17:10] <cronix> mons are running 0.80 whereas osd's are running 0.72
[17:10] * KaZeR (~kazer@64.201.252.132) Quit (Ping timeout: 480 seconds)
[17:11] <blinky_ghost> ghartz: Yes I know, but I was simulating a catastrophic failure, this is a LAB, so not a problem. Currently I have 4 mons servers and I notice that if I stop all the mon daemons except one it doesn't allow me to access ceph cluster status. It trows this -- 192.168.151.1:0/1024635 >> 192.168.151.4:6789/0 pipe(0x7eff740073d0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7eff74005640).fault
[17:11] <blinky_ghost> : Is this normal?
[17:12] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[17:12] <janos_> can't reach quorum, i'd say
[17:13] <seapasulli> has anyone had issues with creating and deleting volumes in cinder with ceph backend?
[17:13] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[17:14] <absynth> cronix: i guess you can't just update the mons?
[17:14] <seapasulli> I created 100 and they create instantly just fine. Deleting them on the other hand deletes around 25-30 of them before it seems to time out
[17:14] <absynth> sorry, the OSDs
[17:14] <absynth> a bit too many of them to upgrade now, in the middle of an issue
[17:14] <cronix> absynth: mhm no
[17:14] <cronix> version mismatch seems to be a diffrent issue
[17:14] <cronix> theyre in this state since nerly a month
[17:14] <absynth> the osds that are down - are the osd processes still running?
[17:14] <cronix> jep
[17:15] <absynth> that's weird
[17:15] <cronix> there running and logging
[17:15] <cronix> those who are down cant be pinged though
[17:15] <KB> blinky_ghost: mons need a quorum of >50%, so if you have 3 mons, you can get quorum with 2, but not 1. with 5, you can get quorum with 3, not 2 or 1. http://ceph.com/docs/master/rados/configuration/mon-config-ref/#monitor-quorum
[17:15] <absynth> can you manually ping other OSDs from the ones that supposedly cnnot reach the others?
[17:15] <cronix> and they come up again after a while
[17:15] <cronix> jep
[17:15] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[17:15] <cronix> via ping
[17:15] <sage> cronix: hi
[17:15] <cronix> i can reach osd hosts
[17:15] <cronix> hi sage
[17:15] <absynth> sage: your customer. ;)
[17:15] <cronix> you might remember me from ceph day in frankfurt at dell
[17:16] <dcurtiss> sage: Who moderates the ceph-users list? I emailed it yesterday at 2pm (since noone answered my question here), but my email is still awaiting approval to be posted.
[17:16] <absynth> that's a quite large testbed setup you have there, if i may say so
[17:16] * haomaiwa_ (~haomaiwan@124.161.72.250) Quit (Remote host closed the connection)
[17:16] <sage> cronix: if you have osds going up/down the quick way to stop the flapping is to 'ceph osd set nodown' until things settle
[17:17] * haomaiwang (~haomaiwan@li634-52.members.linode.com) has joined #ceph
[17:17] <cronix> yeah well i already set the noout, will set nodown aswell
[17:17] <cronix> but that seems to me to just cure the symptome and not the root cause amiright?
[17:17] <absynth> yeah
[17:18] <cronix> its not as big of an issue
[17:18] <sage> yeah, usually it is an asymmetric network issue. or a bug
[17:18] <cronix> since were not live yetz
[17:18] <sage> not many people running 0.72.x.. i can't remember which issues may still be present.
[17:18] <cronix> well talking about bugs
[17:18] <sage> anyway, if with nodown the pgs don't all peer, that suggests there is something prevenitng communication
[17:18] <cronix> weve came accross some weird behaviour with ce4ph and dmcrypt
[17:18] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[17:19] <jpieper> I'm having some unexpected backfill behavior when adding a new OSD. I currently have "osd max backfills = 1", but when I add a new OSD, my "active+remapped+backfilling" count goes up to the number of OSDs in my cluster, as if every OSD is pushing at once to the new arrival. Is there anyway to limit the maximum number of backfills on an OSD at once? v0.80.1
[17:19] <cronix> what i did the last 2 days might have caused this weirdness
[17:19] <sage> cronix: interesting. i'm not sure how many people are using the dm-crypt stuff either :) but very interested in reproducible bug reports
[17:19] <cronix> we had some broken HDD's and our dc run team replaced them
[17:20] * huangjun (~kvirc@117.151.45.241) has joined #ceph
[17:20] <cronix> with ceph-disk zap /dev/device
[17:20] <absynth> sometimes, you see weird issues during backfill
[17:20] <cronix> ceph-disk-prepare --dmcrypt /dev/device
[17:20] <absynth> due to the i/o load chaning
[17:20] <absynth> +g
[17:20] <cronix> and ceph-disk activate /dev/device
[17:20] <cronix> they got assigned the old OSD number
[17:20] <cronix> which was set to out and rm'd before
[17:21] <cronix> as a quick hack i rm'd the auth keys of said osd numbers from ceph and after that the OSD's rejoined the cluster
[17:21] <huangjun> if the data pool flag 'full' was set, the kernel client can also write data?
[17:21] <cronix> as the old osd number that was removed prior to replacing the hdd
[17:21] <huangjun> but use rados to put a object, it return NO SPACE
[17:21] <sage> cronix: ah. the 'ceph osd rm ...' command should possibly remove the associated auth key
[17:22] * jack (~jack@178.250.208.80) Quit (Quit: leaving)
[17:22] <cronix> that did not happen here
[17:22] <sage> huangjun: older versions of the kernel client are less polite in that regard. newer versions will block by default
[17:22] <cronix> but i thought old already used osd numbers wont appear ever again
[17:22] <sage> cronix: if you osd rm ... they will
[17:22] <cronix> that happend again today
[17:23] <cronix> with another osd
[17:23] <cronix> without rm'ing the osd
[17:23] * KaZeR (~kazer@64.201.252.132) has joined #ceph
[17:23] <sage> they are assigned a new unique uuid tho to avoid confusion internally
[17:23] <cronix> anyways
[17:23] <cronix> it wont stabelize atm
[17:23] <cronix> 79% peering state
[17:24] <cronix> sage: this is the current state of the global cluster: http://pastebin.com/index/za5EkxTe
[17:24] * aldavud (~aldavud@wlandevil.nine.ch) has joined #ceph
[17:26] <sage> cronix: pick a random osd that is marked down and see why it isn't starting. (confirm the daemon is running, check /var/log/ceph/ceph-osd.NNN.log on that node for messages, restart it and see if it starts then)
[17:26] <sage> also, upgrade to 0.80.x at your earliest convenience.
[17:27] <sage> emperor is no longer tested or maintained and i have a hard time remembering which bugs were/weren't present
[17:27] * ade (~abradshaw@193.202.255.218) Quit (Remote host closed the connection)
[17:27] <cronix> were planing to asapst
[17:28] <cronix> but first we need to get this storage beast stable again
[17:28] <cronix> :)
[17:28] <huangjun> in ReplicatedPG, it checked the full flag, so it worked in rados/rbd, but why fs doesn't work?
[17:28] <dcurtiss> does the swift API work in Firefly? I can't get it to authenticate me
[17:28] <huangjun> anything related to kclient's async write?
[17:29] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) Quit (Ping timeout: 480 seconds)
[17:30] <cronix> http://pastebin.com/LRqTDDHT
[17:30] <cronix> thats an example logfile content of a "down" osd
[17:30] <cronix> i have no idea what that means
[17:32] * shang (~ShangWu@ipvpn110138.netvigator.com) Quit (Quit: Ex-Chat)
[17:32] * haomaiwa_ (~haomaiwan@124.248.205.17) has joined #ceph
[17:32] <cronix> but were down to 14% peering PG's again
[17:32] <cronix> sage: poke ;)
[17:33] <cronix> we also have a lot of this entrys in our osd logs:
[17:33] <cronix> 2014-06-05 17:00:15.191005 7f4702411700 0 -- 10.78.6.138:0/13611 >> 10.78.7.138:6807/10015160 pipe(0x284d1400 sd=112 :56587 s=1 pgs=0 cs=0 l=1 c=0x380cd1e0).connect claims to be 10.78.7.138:6807/13048581 not 10.78.7.138:6807/10015160 - wrong node!
[17:34] * yuriw (~Adium@ABordeaux-654-1-80-223.w109-214.abo.wanadoo.fr) has joined #ceph
[17:35] * Meths_ (~meths@2.25.213.253) has joined #ceph
[17:37] * Meths (~meths@2.25.211.246) Quit (Read error: Operation timed out)
[17:38] * lalatenduM (~lalatendu@nat-pool-blr-t.redhat.com) Quit (Quit: Leaving)
[17:39] * haomaiwang (~haomaiwan@li634-52.members.linode.com) Quit (Ping timeout: 480 seconds)
[17:39] * odyssey4me_ is now known as odyssey4me
[17:40] * newbie|2 (~kvirc@117.151.45.241) has joined #ceph
[17:40] * odyssey4me (~odyssey4m@165.233.71.2) Quit (Quit: Leaving)
[17:40] * talonisx (~talonisx@pool-108-18-97-131.washdc.fios.verizon.net) has joined #ceph
[17:41] * odyssey4me (~odyssey4m@165.233.71.2) has joined #ceph
[17:41] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[17:43] * huangjun (~kvirc@117.151.45.241) Quit (Ping timeout: 480 seconds)
[17:43] * ajazdzewski (~quassel@lpz-66.sprd.net) Quit (Remote host closed the connection)
[17:44] * rturk|afk is now known as rturk
[17:45] * rmoe (~quassel@12.164.168.117) has joined #ceph
[17:45] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) has joined #ceph
[17:46] * zerick (~eocrospom@190.118.43.113) Quit (Read error: Connection reset by peer)
[17:46] <sage> cronix: that looks like a bug we fixed a while back but didn't backport
[17:46] <cronix> ok so basically a upgrade to firefly should stablize the cluster?
[17:46] <sage> if your mons are already firefly i would upgrade the packages on the osds too
[17:46] <sage> it will at least get you past that issue :)
[17:47] <sage> hopefully that is the only problem.
[17:47] <cronix> mhmok
[17:47] <sage> the 'wrong node' messages are a side-effect of the nodown flag being set
[17:47] <cronix> well atm 2 mons are 0.80 and the other 3 are 0.72
[17:47] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[17:48] <sage> ah, should upgrade the remaining mons first
[17:49] <cronix> sure
[17:50] <cronix> okay
[17:50] * hasues (~hasues@kwfw01.scrippsnetworksinteractive.com) has joined #ceph
[17:50] <cronix> thanks for the input
[17:51] <cronix> we will have our dev's have a look tomorrow and then decide how we will proceed
[17:51] <cronix> have a nice day, it's end of todays work for me
[17:52] <cronix> ttyl8r
[17:53] * odyssey4me_ (~odyssey4m@165.233.71.2) has joined #ceph
[17:53] * dignus (~jkooijman@t-x.dignus.nl) has joined #ceph
[17:54] <jpieper> I'll repeat my question because it may have gotten lost in some chatter: I'm having some unexpected backfill behavior when adding a new OSD. I currently have "osd max backfills = 1", but when I add a new OSD, my "active+remapped+backfilling" count goes up to the number of OSDs in my cluster, as if every OSD is pushing at once to the new arrival. Is there anyway to limit the maximum number of backfills on an OSD at once?
[17:54] <jpieper> v0.80.1
[17:56] <newbie|2> jpieper: a new osd need to store data from exists osds, so there are backfills
[17:58] * ScOut3R (~ScOut3R@catv-80-99-64-8.catv.broadband.hu) Quit (Ping timeout: 480 seconds)
[17:58] <jpieper> newbie|2: Yes I know, however my understanding was that there was a reservation system such that only some of the OSDs could push a backfill at once. I don't have sufficient IOPs on the cluster to support every OSD backfilling at the same time, which is why I have "max backfills" set lower than default.
[17:59] * odyssey4me (~odyssey4m@165.233.71.2) Quit (Ping timeout: 480 seconds)
[18:00] * nwat (~textual@eduroam-248-28.ucsc.edu) has joined #ceph
[18:01] * newbie|2 is now known as huangjun
[18:04] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[18:04] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[18:04] * odyssey4me (~odyssey4m@165.233.71.2) has joined #ceph
[18:05] * angdraug (~angdraug@12.164.168.117) has joined #ceph
[18:06] * Cube (~Cube@66-87-66-229.pools.spcsdns.net) has joined #ceph
[18:06] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[18:07] * odyssey4me_ (~odyssey4m@165.233.71.2) Quit (Read error: Operation timed out)
[18:07] * dcurtiss (~dcurtiss@130.164.62.72) Quit (Read error: Connection reset by peer)
[18:08] <Karcaw> is there a way to delete a pool by number? Somehow i got one with no name, but all the tools seem to want a name, but it i give it '' on the command line it ignores me
[18:09] * dcurtiss (~dcurtiss@130.164.62.72) has joined #ceph
[18:10] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[18:11] <dcurtiss> jpieper: make sure your osd_max_backfills setting is live on the server:
[18:11] <dcurtiss> ceph --admin-daemon /var/run/ceph/ceph-client*.asok config show | grep osd_max_backfills
[18:12] * xarses (~andreww@12.164.168.117) has joined #ceph
[18:12] <huangjun> Karcaw: can you eacape it , use rados rmpool \"
[18:12] * ksperis (~ksperis@46.218.42.103) Quit (Quit: Quitte)
[18:13] * kwaegema (~kwaegema@daenerys.ugent.be) Quit (Ping timeout: 480 seconds)
[18:14] <Karcaw> so rados rmpool took the double '' '' for blanks whereas ceph osd pool rm did not.. thanks
[18:16] * ade (~abradshaw@80-72-52-4.cmts.powersurf.li) has joined #ceph
[18:16] <jpieper> dcurtiss: Sure enough, the offending OSD had the wrong configuration (all the others were correct). I'm guessing it must not have read the value correctly from the ceph.conf and the others had it from a previous injectargs.
[18:17] * drankis_ (~drankis__@37.148.173.239) has joined #ceph
[18:17] <dcurtiss> you can change it live if you want: http://ceph.com/docs/master/rados/configuration/ceph-conf/#runtime-changes
[18:17] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[18:17] * madkiss (~madkiss@2001:6f8:12c3:f00f:d479:dc1d:69fc:41dd) Quit (Ping timeout: 480 seconds)
[18:18] <sage> jpieper: ceph pg dump | grep backfill and see if they are all backfilling to the same osd. ideally they are not
[18:18] <jpieper> dcurtiss: Yep, I already did. I'm just wondering why it didn't get the correct value on start. In the ceph.conf I have "[osd]\n max backfills = 1". Isn't that supposed to be the same parameter?
[18:18] <sage> jpieper: oh, nevermind :)
[18:18] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[18:19] <dcurtiss> No idea... I'm still new to ceph, and have my own issues. Speaking of which, mine seems to have gotten lost in the chatter, too:
[18:19] <dcurtiss> Does the swift API work in Firefly? I can't get it to authenticate me.
[18:20] <huangjun> sage: why the fs client have different behave with rbd/rados? i think they all send MOSDOp msg to OSD, and osd check the pool full flag, so rados can not write a full pool
[18:21] * rejy (~rmc@nat-pool-blr-t.redhat.com) Quit (Quit: Leaving)
[18:22] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[18:22] * sprachgenerator (~sprachgen@173.150.246.128) has joined #ceph
[18:24] * blSnoopy (~snoopy@miram.persei.mw.lg.virgo.supercluster.net) has joined #ceph
[18:25] * rturk is now known as rturk|afk
[18:28] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[18:31] * sprachgenerator_ (~sprachgen@173.150.246.128) has joined #ceph
[18:31] * madkiss (~madkiss@2001:6f8:12c3:f00f:ed04:a645:694a:4b57) has joined #ceph
[18:32] * asfsa (~peng@119.2.3.164) has joined #ceph
[18:32] <sage> huangjun: it's not an fs vs rbd think, but a kernel client implementation vs librados thing.
[18:33] <sage> joshd knows more; i forget if the older kernel didn't observe full flag at all or if it was a more subtle problem. in any case, a recent kernel should work properly.
[18:34] * sprachgenerator_ (~sprachgen@173.150.246.128) Quit (Read error: Connection reset by peer)
[18:34] * sprachgenerator (~sprachgen@173.150.246.128) Quit (Read error: Connection reset by peer)
[18:35] <huangjun> uhh, i think all write request will handle in ReplicatedPG, and if it checked the flag, then kernel client and librados should get the same result
[18:36] <huangjun> i'll try the newest kernel clent
[18:36] * madkiss (~madkiss@2001:6f8:12c3:f00f:ed04:a645:694a:4b57) Quit ()
[18:38] * topro (~prousa@host-62-245-142-50.customer.m-online.net) Quit (Quit: Konversation terminated!)
[18:39] * ade (~abradshaw@80-72-52-4.cmts.powersurf.li) Quit (Quit: Too sexy for his shirt)
[18:39] * sprachgenerator (~sprachgen@173.150.176.52) has joined #ceph
[18:40] * asfsa (~peng@119.2.3.164) Quit (Ping timeout: 480 seconds)
[18:41] * gregsfortytwo (~Adium@129.210.115.14) Quit (Quit: Leaving.)
[18:41] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[18:42] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[18:43] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[18:43] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) has joined #ceph
[18:43] * sprachgenerator (~sprachgen@173.150.176.52) Quit ()
[18:43] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit ()
[18:44] * michalefty (~micha@188-195-129-145-dynip.superkabel.de) has joined #ceph
[18:45] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has joined #ceph
[18:47] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) Quit ()
[18:49] * m0e (~Moe@41.45.107.223) has joined #ceph
[18:50] * huangjun (~kvirc@117.151.45.241) Quit (Ping timeout: 480 seconds)
[18:50] * alfredodeza (~alfredode@198.206.133.89) has joined #ceph
[18:53] <kfei> I deployed a Win7 VM with QEMU-RBD disk by libvirt.
[18:53] <kfei> When monitoring outputs of `ceph -w`, I find that when I boot the Win7 VM up, there are about consecutive 3 minutes that `ceph -w` gives me a interesting log: "... 567 KB/s rd, 567 op/s", "... 789 KB/s rd, 789 op/s" and so on.
[18:53] <kfei> Note that the number of rps is always the same as the number of ops, which shows every operation is nearly 1KB.
[18:53] <kfei> I just can't understand what this means, does it relate to some I/O patterns during the Win7 VM booting process?
[18:54] * mjeanson_ (~mjeanson@bell.multivax.ca) has joined #ceph
[18:54] * mjeanson (~mjeanson@00012705.user.oftc.net) Quit (Ping timeout: 480 seconds)
[18:57] * thb (~me@0001bd58.user.oftc.net) Quit (Quit: Leaving.)
[18:58] * haomaiwa_ (~haomaiwan@124.248.205.17) Quit (Remote host closed the connection)
[18:58] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[19:01] * bandrus (~Adium@adsl-75-5-249-229.dsl.scrm01.sbcglobal.net) Quit (Quit: Leaving.)
[19:03] * sjusthm (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) has joined #ceph
[19:03] * sm1ly (~sm1ly@ppp109-252-169-241.pppoe.spdop.ru) Quit (Read error: Connection reset by peer)
[19:03] * sm1ly (~sm1ly@ppp109-252-169-241.pppoe.spdop.ru) has joined #ceph
[19:09] * gregsfortytwo (~Adium@129.210.115.14) Quit (Quit: Leaving.)
[19:11] * bandrus (~Adium@66.87.118.131) has joined #ceph
[19:13] <ponyofdeath> anyone running an mysql database on a ceph cluster? it seems super slow to import a db. maby the write pattern kvm is saying the disk is 100% utilized
[19:15] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) has joined #ceph
[19:16] <cookednoodles> what setup ?
[19:17] * talonisx (~talonisx@pool-108-18-97-131.washdc.fios.verizon.net) Quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/)
[19:18] * reed (~reed@75-101-54-131.dsl.static.sonic.net) has joined #ceph
[19:18] <ponyofdeath> cookednoodles: 32 osd's 3TB each with 100GB ssd per osd for journal
[19:18] * xdeller_ (~xdeller@h195-91-128-218.ln.rinet.ru) has joined #ceph
[19:30] * michalefty (~micha@188-195-129-145-dynip.superkabel.de) Quit (Quit: Leaving.)
[19:31] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[19:31] * markbby (~Adium@168.94.245.1) Quit (Quit: Leaving.)
[19:33] * Nacer_ (~Nacer@194-87-190-213.intermediasud.com) has joined #ceph
[19:34] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) Quit (Quit: koleosfuscus)
[19:36] * Nacer (~Nacer@252-87-190-213.intermediasud.com) Quit (Read error: Operation timed out)
[19:36] <wrencsok> we have perf issues with sql type workflows via rbd. its the one area our cluster does not perform well, partly based off how ceph works. that would change a lot with ssd based pools, but.. we don't have those in production yet. did you try enabling batched operations and query caching? also it may make a difference as far as which storage engine you use myisam vs innodb. personally, I haven't gotten a chance to test those things on that
[19:37] * rturk|afk is now known as rturk
[19:37] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[19:38] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[19:38] * ChanServ sets mode +v andreask
[19:38] * dosaboy (~dosaboy@65.93.189.91.lcy-01.canonistack.canonical.com) Quit (Quit: leaving)
[19:39] * amospalla (~amospalla@0001a39c.user.oftc.net) Quit (Quit: WeeChat 0.4.3)
[19:41] * dosaboy (~dosaboy@65.93.189.91.lcy-01.canonistack.canonical.com) has joined #ceph
[19:41] <ponyofdeath> wrencsok: well an simple 17GB import takes more then a day
[19:41] * Nacer_ (~Nacer@194-87-190-213.intermediasud.com) Quit (Ping timeout: 480 seconds)
[19:43] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[19:43] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) has joined #ceph
[19:44] * sep (~sep@2a04:2740:1:0:52e5:49ff:feeb:32) has joined #ceph
[19:45] <ponyofdeath> http://bpaste.net/show/fQucjcHXZhzcTNYR2UJJ/
[19:45] <ponyofdeath> wrencsok: those are my bench results
[19:45] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) Quit (Quit: Leaving.)
[19:47] <ponyofdeath> wrencsok: also what are ur rbd cache settings in kvm?
[19:47] <ponyofdeath> wrencsok: and u are using virtio
[19:47] * fford (~wee@p4FC9DCA9.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[19:48] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[19:49] * drankis_ (~drankis__@37.148.173.239) Quit (Ping timeout: 480 seconds)
[19:52] * gregsfortytwo (~Adium@129.210.115.14) Quit (Quit: Leaving.)
[19:55] * sputnik13 (~sputnik13@207.8.121.241) has joined #ceph
[19:55] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[19:56] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) Quit (Quit: Leaving.)
[19:56] * jpieper (~jpieper@173-9-58-170-NewEngland.hfc.comcastbusiness.net) Quit (Quit: Ex-Chat)
[19:56] * aldavud (~aldavud@wlandevil.nine.ch) Quit (Ping timeout: 480 seconds)
[19:57] * funnel (~funnel@0001c7d4.user.oftc.net) Quit (Remote host closed the connection)
[19:57] * drankis_ (~drankis__@89.111.13.198) has joined #ceph
[19:58] * amospalla (~amospalla@0001a39c.user.oftc.net) has joined #ceph
[19:59] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) has joined #ceph
[19:59] * Meths_ is now known as Meths
[19:59] * ganders (~root@200-127-158-54.net.prima.net.ar) Quit (Quit: WeeChat 0.4.1)
[20:00] * ganders (~root@200-127-158-54.net.prima.net.ar) has joined #ceph
[20:00] * dxd828 (~dxd828@dsl-dynamic-77-44-45-38.interdsl.co.uk) has joined #ceph
[20:00] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) has joined #ceph
[20:02] * odyssey4me (~odyssey4m@165.233.71.2) Quit (Quit: Leaving)
[20:03] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[20:06] * ajazdzewski (~quassel@2001:4dd0:ff00:9081:9934:7752:90d8:e7b7) has joined #ceph
[20:08] * zack_dolby (~textual@pdf8519e7.tokynt01.ap.so-net.ne.jp) has joined #ceph
[20:13] * ircolle (~Adium@mobile-166-137-217-150.mycingular.net) has joined #ceph
[20:15] * reed (~reed@75-101-54-131.dsl.static.sonic.net) Quit (Remote host closed the connection)
[20:16] * reed (~reed@75-101-54-131.dsl.static.sonic.net) has joined #ceph
[20:16] * gregsfortytwo (~Adium@129.210.115.14) Quit (Quit: Leaving.)
[20:16] <KaZeR> i added an OSD to a running cluster and the recovery is really, really slow. It has been running for > 24h now for a 7TB cluster
[20:16] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[20:16] <KaZeR> any hint on how i can speed this up?
[20:17] <KaZeR> over the last hour the recovery is around 5 objects/second
[20:17] <KaZeR> i still have ~10.000 objects to recover
[20:17] <sep> how many disks are reccomended for ceph osd machines ? i notice supermicro wants to use 72 drives+ 12 ssd's in one machine. Is that a sane configuration on ceph ? i belive i once read something about 12, but my memory might be failing or standards might have changed? ;; http://www.supermicro.nl/products/nfo/storage_ceph.cfm
[20:17] * dxd828_ (~dxd828@dsl-dynamic-77-44-45-38.interdsl.co.uk) has joined #ceph
[20:17] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[20:18] * markbby (~Adium@168.94.245.2) has joined #ceph
[20:18] * rturk is now known as rturk|afk
[20:18] * rturk|afk is now known as rturk
[20:19] * gregsfortytwo (~Adium@129.210.115.14) Quit ()
[20:19] * The_Bishop (~bishop@f055213012.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[20:20] * dxd828 (~dxd828@dsl-dynamic-77-44-45-38.interdsl.co.uk) Quit (Ping timeout: 480 seconds)
[20:21] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[20:21] * sarob (~sarob@2001:4998:effd:600:1d59:db5a:7e57:5a18) has joined #ceph
[20:24] <Pauline> well, there are others on that page: SSG-6027R-OSD040H
[20:24] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) Quit (Quit: koleosfuscus)
[20:25] * sputnik13 (~sputnik13@207.8.121.241) Quit (Quit: Textual IRC Client: www.textualapp.com)
[20:26] <Pauline> though they spec a wee bit high. guess they wanna sell you expensive machines. 128GB for 12 osds...
[20:27] * The_Bishop (~bishop@f055213012.adsl.alicedsl.de) has joined #ceph
[20:27] * sarob (~sarob@2001:4998:effd:600:1d59:db5a:7e57:5a18) Quit (Remote host closed the connection)
[20:28] <janos_> lol
[20:28] * sarob (~sarob@nat-dip27-wl-a.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:29] * sarob_ (~sarob@2001:4998:effd:600:3524:b041:662d:d394) has joined #ceph
[20:29] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) has joined #ceph
[20:30] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has joined #ceph
[20:31] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has left #ceph
[20:31] * sarob_ (~sarob@2001:4998:effd:600:3524:b041:662d:d394) Quit (Remote host closed the connection)
[20:31] * sarob_ (~sarob@nat-dip27-wl-a.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:32] * sarob__ (~sarob@2001:4998:effd:600:c35:c034:c35c:7034) has joined #ceph
[20:34] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) Quit (Quit: Leaving.)
[20:34] <sep> well that machine have 12 drives on 2u. while the large one have 72drives on 4u ; saves a lots of space. ofcourse something like SSG-5017R-iHDP 12 drives on 1 u could perhaps be used
[20:36] * sarob (~sarob@nat-dip27-wl-a.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[20:37] <KaZeR> can someone enlight me about defining an optimal PG number ?
[20:37] * aldavud (~aldavud@213.55.184.239) has joined #ceph
[20:38] <Pauline> sep: the problem is that you do not want too many disks in one box. box fails > lots of rebalancing
[20:38] * rturk is now known as rturk|afk
[20:38] <Pauline> 72 drives is outside my comfort zone
[20:39] * thb (~me@port-30786.pppoe.wtnet.de) has joined #ceph
[20:39] * sarob_ (~sarob@nat-dip27-wl-a.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[20:39] * thb is now known as Guest12683
[20:39] <Pauline> then again, if you have enough 72 drive units, you're fine too ^^
[20:40] * Guest12683 is now known as thb
[20:40] * markbby (~Adium@168.94.245.2) Quit (Remote host closed the connection)
[20:41] <dcurtiss> Does the swift API work in v0.80.1? I can't get it to authenticate me.
[20:47] * rendar (~I@host123-161-dynamic.1-87-r.retail.telecomitalia.it) Quit (Read error: Operation timed out)
[20:47] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[20:47] * nwat (~textual@eduroam-248-28.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[20:48] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[20:50] <wrencsok> ponyofdeath: in production i don't use rbd caching atm. enabling that in the future tho. in lab i use it in writethrough mode and see significant per f improvements. and yes we use virtio.
[20:51] <ponyofdeath> wrencsok what are ur cache settings int /etc/ceph/ceph.conf on the kvm hosts
[20:53] <ponyofdeath> wrencsok: how do u seed the db's? and how big are the databaes?
[20:53] <ponyofdeath> wrencsok: we have a 17GB db dump and its on day 2 now of import
[20:54] <ponyofdeath> the kvm vdb is 100% utilized but with no solid write or read metrics from iostat
[20:54] <wrencsok> wow
[20:54] <ponyofdeath> yeah :)
[20:54] * rturk|afk is now known as rturk
[20:54] <ponyofdeath> and do u use xfs for ur osd?s
[20:55] <wrencsok> this is my lab parent ceph.conf. i set other things in the guest config.xml. sorry about the slight channel spam to come. its short tho.
[20:55] <wrencsok> log_max_recent = 500
[20:55] <wrencsok> rbd_cache = true
[20:55] <wrencsok> rbd_cache_max_dirty = 0
[20:55] <wrencsok> rbd_cache_writethrough_until_flush = true
[20:55] <wrencsok> admin socket =/tmp/rbd-$name.sock
[20:55] <ponyofdeath> wrencsok: hmm ok thats diff then mine
[20:56] <sep> Pauline, thanks for your input- the rebalancing issue realy makes sense.
[20:56] <ponyofdeath> http://bpaste.net/show/V6LQkkaIDhos1fUsxIkr/
[20:56] <ponyofdeath> wrencsok: is mine
[20:56] <ponyofdeath> why did u go with max dirty = 0
[20:57] <ponyofdeath> Start out in write-through mode, and switch to write-back after the first flush request is received. Enabling this is a conservative but safe setting in case VMs running on rbd are too old to send flushes, like the virtio driver in Linux before 2.6.32.
[20:57] <wrencsok> writethrough mdoe
[20:58] <ponyofdeath> ahh ok
[20:58] <wrencsok> headding out pony, need to get away from office paint smell.
[20:58] <ponyofdeath> i use cache='writeback'
[20:58] <ponyofdeath> haha ok thanks for u help
[20:58] * sarob__ (~sarob@2001:4998:effd:600:c35:c034:c35c:7034) Quit (Remote host closed the connection)
[20:58] <wrencsok> one thing that may really boost sql for you
[20:58] <wrencsok> is a high iops disk.
[20:58] <wrencsok> like use a ssd based pool.
[20:59] <ponyofdeath> yeah
[20:59] <wrencsok> because ceph objects are limited by the slowest disk
[20:59] * sarob (~sarob@nat-dip27-wl-a.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:59] <ponyofdeath> thats prob next investment is ssd only ceph servers
[20:59] <wrencsok> so your disk in a set of replicas may support say 150 iops. that's all that user will really see without caching.
[20:59] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[21:00] <wrencsok> we're going that route soon, ourselves.
[21:00] * dxd828_ (~dxd828@dsl-dynamic-77-44-45-38.interdsl.co.uk) Quit (Quit: Textual IRC Client: www.textualapp.com)
[21:00] <wrencsok> anyhow havce to head out eyes are burning from paint. good luck
[21:00] <ponyofdeath> thanks man
[21:00] * sarob_ (~sarob@2001:4998:effd:600:712a:f1ab:5f46:5ab5) has joined #ceph
[21:01] * scuttlemonkey (~scuttlemo@72.11.211.243) has joined #ceph
[21:01] * ChanServ sets mode +o scuttlemonkey
[21:05] * sputnik13 (~sputnik13@207.8.121.241) has joined #ceph
[21:05] * sputnik13 (~sputnik13@207.8.121.241) Quit ()
[21:07] * sarob (~sarob@nat-dip27-wl-a.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[21:07] * sputnik13 (~sputnik13@207.8.121.241) has joined #ceph
[21:11] * steki (~steki@cable-94-189-165-169.dynamic.sbb.rs) has joined #ceph
[21:11] * sarob_ (~sarob@2001:4998:effd:600:712a:f1ab:5f46:5ab5) Quit (Remote host closed the connection)
[21:11] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) Quit (Quit: Leaving.)
[21:11] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[21:14] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[21:17] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[21:18] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) has joined #ceph
[21:18] * zidarsk81 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[21:18] * zidarsk81 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[21:19] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) has joined #ceph
[21:19] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[21:21] * sarob (~sarob@2001:4998:effd:600:b55e:6237:8eea:4a1e) has joined #ceph
[21:25] * stewiem2000 (~stewiem20@195.10.250.233) Quit (Read error: Operation timed out)
[21:25] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) Quit (Ping timeout: 480 seconds)
[21:25] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[21:25] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) has joined #ceph
[21:27] * stewiem2000 (~stewiem20@195.10.250.233) has joined #ceph
[21:28] * rotbeard (~redbeard@2a02:908:df11:9480:76f0:6dff:fe3b:994d) has joined #ceph
[21:32] * baylight (~tbayly@69-195-66-4.unifiedlayer.com) Quit (Ping timeout: 480 seconds)
[21:32] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[21:35] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) Quit (Quit: koleosfuscus)
[21:35] * fghaas (~florian@91-119-141-13.dynamic.xdsl-line.inode.at) has joined #ceph
[21:36] * hijacker (~hijacker@bgva.sonic.taxback.ess.ie) Quit (Ping timeout: 480 seconds)
[21:38] * blinky_ghost (~psousa@213.228.167.67) Quit (Read error: Operation timed out)
[21:39] * TMM (~hp@178-84-46-106.dynamic.upc.nl) has joined #ceph
[21:43] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) has joined #ceph
[21:43] * thomnico (~thomnico@2a01:e35:8b41:120:b841:4631:c163:3a27) Quit (Quit: Ex-Chat)
[21:56] * fghaas (~florian@91-119-141-13.dynamic.xdsl-line.inode.at) has left #ceph
[21:56] * drankis_ (~drankis__@89.111.13.198) Quit (Remote host closed the connection)
[21:57] * drankis (~drankis__@37.148.173.239) has joined #ceph
[21:58] * ajazdzewski (~quassel@2001:4dd0:ff00:9081:9934:7752:90d8:e7b7) Quit (Ping timeout: 480 seconds)
[21:58] * hijacker (~hijacker@bgva.sonic.taxback.ess.ie) has joined #ceph
[21:58] * nwat (~textual@eduroam-248-28.ucsc.edu) has joined #ceph
[21:59] * aldavud (~aldavud@213.55.184.239) Quit (Ping timeout: 480 seconds)
[22:03] * aldavud (~aldavud@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[22:11] * rturk is now known as rturk|afk
[22:13] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[22:13] * ChanServ sets mode +v andreask
[22:13] * rturk|afk is now known as rturk
[22:13] * sroy (~sroy@2607:fad8:4:6:3e97:eff:feb5:1e2b) Quit (Quit: Quitte)
[22:14] * yuriw (~Adium@ABordeaux-654-1-80-223.w109-214.abo.wanadoo.fr) Quit (Remote host closed the connection)
[22:15] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[22:16] * bandrus1 (~Adium@66-87-118-119.pools.spcsdns.net) has joined #ceph
[22:19] * rturk is now known as rturk|afk
[22:20] * rturk|afk is now known as rturk
[22:20] * bandrus (~Adium@66.87.118.131) Quit (Ping timeout: 480 seconds)
[22:21] * sleinen (~Adium@2001:620:0:26:a4ec:c48f:b068:683d) has joined #ceph
[22:25] * bandrus1 (~Adium@66-87-118-119.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[22:25] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) Quit (Quit: Leaving.)
[22:26] * rwheeler (~rwheeler@nat-pool-bos-t.redhat.com) Quit (Quit: Leaving)
[22:27] * stewiem2000 (~stewiem20@195.10.250.233) Quit (Read error: Connection timed out)
[22:27] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[22:29] * scuttlemonkey (~scuttlemo@72.11.211.243) Quit (Ping timeout: 480 seconds)
[22:29] * stewiem2000 (~stewiem20@195.10.250.233) has joined #ceph
[22:32] * baylight (~tbayly@69-195-66-4.unifiedlayer.com) has joined #ceph
[22:33] * Tamil (~Adium@cpe-142-136-97-92.socal.res.rr.com) has joined #ceph
[22:33] * gregsfortytwo (~Adium@129.210.115.14) Quit (Read error: Connection reset by peer)
[22:33] * gregsfortytwo (~Adium@129.210.115.14) has joined #ceph
[22:34] * ganders (~root@200-127-158-54.net.prima.net.ar) Quit (Quit: WeeChat 0.4.1)
[22:34] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[22:36] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) Quit (Quit: Leaving.)
[22:43] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[22:43] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) Quit (Quit: Konversation terminated!)
[22:45] * stewiem2000 (~stewiem20@195.10.250.233) Quit (Read error: Connection reset by peer)
[22:45] * stewiem2000 (~stewiem20@195.10.250.233) has joined #ceph
[22:47] * bandrus (~Adium@66-87-119-105.pools.spcsdns.net) has joined #ceph
[22:47] * danieagle (~Daniel@191.250.136.251) has joined #ceph
[22:48] * allsystemsarego (~allsystem@188.27.188.69) Quit (Quit: Leaving)
[22:53] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[22:54] * analbeard (~shw@host86-155-192-138.range86-155.btcentralplus.com) has joined #ceph
[22:56] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) Quit (Quit: Leaving.)
[22:57] * nwat (~textual@eduroam-248-28.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[22:57] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[22:58] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Remote host closed the connection)
[22:58] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[22:59] * gregsfortytwo (~Adium@129.210.115.14) Quit (Quit: Leaving.)
[23:01] * jcsp1 (~Adium@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[23:03] * ajazdzewski (~quassel@2001:4dd0:ff00:9081:9934:7752:90d8:e7b7) has joined #ceph
[23:03] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[23:04] * jcsp (~Adium@0001bf3a.user.oftc.net) Quit (Ping timeout: 480 seconds)
[23:05] * xarses (~andreww@12.164.168.117) Quit (Ping timeout: 480 seconds)
[23:05] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) Quit (Quit: Leaving)
[23:09] * ircolle (~Adium@mobile-166-137-217-150.mycingular.net) Quit (Read error: Connection reset by peer)
[23:09] * ircolle (~Adium@mobile-166-137-217-150.mycingular.net) has joined #ceph
[23:10] <dcurtiss> How long should I expect to wait for my email to ceph-users to go through? It's been awaiting moderator approval for over a day now.
[23:14] * ajazdzewski (~quassel@2001:4dd0:ff00:9081:9934:7752:90d8:e7b7) Quit (Ping timeout: 480 seconds)
[23:16] * xarses (~andreww@12.164.168.117) has joined #ceph
[23:17] * sarob (~sarob@2001:4998:effd:600:b55e:6237:8eea:4a1e) Quit (Remote host closed the connection)
[23:17] * drankis (~drankis__@37.148.173.239) Quit (Ping timeout: 480 seconds)
[23:18] * sarob (~sarob@2001:4998:effd:600:b55e:6237:8eea:4a1e) has joined #ceph
[23:19] * kevinc (~kevinc__@client65-78.sdsc.edu) Quit (Quit: This computer has gone to sleep)
[23:19] * kevinc (~kevinc__@client65-78.sdsc.edu) has joined #ceph
[23:26] * sarob (~sarob@2001:4998:effd:600:b55e:6237:8eea:4a1e) Quit (Ping timeout: 480 seconds)
[23:26] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[23:28] * rendar (~I@host123-161-dynamic.1-87-r.retail.telecomitalia.it) has joined #ceph
[23:35] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Remote host closed the connection)
[23:35] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[23:40] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[23:43] * davidzlap (~Adium@ip68-4-173-198.oc.oc.cox.net) has joined #ceph
[23:43] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[23:45] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Read error: Operation timed out)
[23:46] * yanfali_lap (~yanfali@75-101-14-52.static.sonic.net) has joined #ceph
[23:48] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[23:53] * sprachgenerator (~sprachgen@173.150.196.199) has joined #ceph
[23:59] * koleosfuscus (~koleosfus@adsl-84-226-68-69.adslplus.ch) Quit (Quit: koleosfuscus)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.