#ceph IRC Log

Index

IRC Log for 2015-12-03

Timestamps are in GMT/BST.

[0:01] * stiopa (~stiopa@cpc73828-dals21-2-0-cust630.20-2.cable.virginm.net) has joined #ceph
[0:01] <dmatson> as a sanity check to preventing clients from stomping on each others toes it seems valuable to me
[0:02] <KaneK> dmatson: as I understand mapping is completely local to a node? I.e. if node dies, there is no any state left on ceph side?
[0:02] <rkeene> Yes, right now I handle locking by adding 2 locks to an image and have threads compete over removals
[0:03] <rkeene> KaneK, No, locks are global -- which is the problem
[0:03] * yguang11 (~yguang11@2001:4998:effd:600:c054:5cdd:9823:962b) Quit (Remote host closed the connection)
[0:03] * yguang11 (~yguang11@2001:4998:effd:600:c054:5cdd:9823:962b) has joined #ceph
[0:04] <KaneK> yeah locks are global, what about image mapping? rbd showmapped shows only images mapped on the node it???s executed on, I assume there is no any state in ceph about rbd maps, so no way to get global mapping information across all nodes?
[0:07] <dmatson> you could use lock_exclusive and pass the hostname possibly? if the same host wants the lock then you could break it and re-lock
[0:08] <KaneK> yeah, that???s most likely how I gonna do that
[0:08] <rkeene> The problem is if the host that was mapping the image goes down, nothing else will know
[0:08] <rkeene> So sure it will be exclusive in that no more than 1 system will use it, but it may be that 0 systems end up using it as a result
[0:09] <dmatson> yeah.. bit of a manual cleanup at that point
[0:09] * EinstCrazy (~EinstCraz@117.13.201.130) has joined #ceph
[0:09] <dmatson> but as you said at least you know the rbd is clean
[0:09] * carlosmalt (~oftc-webi@eduroam-169-233-253-126.ucsc.edu) has joined #ceph
[0:09] * leseb_ is now known as leseb_away
[0:09] * carlosmalt (~oftc-webi@eduroam-169-233-253-126.ucsc.edu) Quit ()
[0:10] <rkeene> Well, unless you manually clean it up wrong
[0:10] <rkeene> (Which given enough time, you will)
[0:11] * t4nk596 (~oftc-webi@192-41-52-12.c7dc.com) has joined #ceph
[0:12] <rkeene> Tedious things that must be done reliably are best left to computers
[0:13] * alram (~alram@cpe-172-250-2-46.socal.res.rr.com) has joined #ceph
[0:15] * chris_333 (~chris.der@192.41.52.12) has joined #ceph
[0:17] * chris_333 (~chris.der@192.41.52.12) has left #ceph
[0:17] * EinstCrazy (~EinstCraz@117.13.201.130) Quit (Ping timeout: 480 seconds)
[0:18] * fred`` (fred@earthli.ng) Quit (Quit: +++ATH0)
[0:18] * cderrick (~chris.der@192.41.52.12) has joined #ceph
[0:19] * malevolent (~quassel@119.Red-79-159-145.dynamicIP.rima-tde.net) Quit (Quit: No Ping reply in 180 seconds.)
[0:19] * hroussea (~hroussea@000200d7.user.oftc.net) Quit (Remote host closed the connection)
[0:19] * Hazelesque (~hazel@2a03:9800:10:13::2) Quit (Remote host closed the connection)
[0:19] * Hazelesque (~hazel@2a03:9800:10:13::2) has joined #ceph
[0:19] * hroussea (~hroussea@2001:bc8:30fa:dfee:8427:a0ff:fe07:3921) has joined #ceph
[0:20] * cderrick (~chris.der@192.41.52.12) has left #ceph
[0:20] * malevolent (~quassel@119.Red-79-159-145.dynamicIP.rima-tde.net) has joined #ceph
[0:21] * wonko_be_ (bernard@november.openminds.be) Quit (Remote host closed the connection)
[0:21] * baffle (baffle@jump.stenstad.net) Quit (Remote host closed the connection)
[0:21] * tdb (~tdb@myrtle.kent.ac.uk) Quit (Remote host closed the connection)
[0:21] * tdb (~tdb@myrtle.kent.ac.uk) has joined #ceph
[0:21] * via (~via@smtp2.matthewvia.info) Quit (Remote host closed the connection)
[0:22] * via (~via@smtp2.matthewvia.info) has joined #ceph
[0:22] * wonko_be (bernard@november.openminds.be) has joined #ceph
[0:25] * ira (~ira@c-73-238-173-100.hsd1.ma.comcast.net) Quit (Quit: Leaving)
[0:25] <joshd> that's where the 'ceph osd blacklist' command helps - you can make sure the 'dead' node can't do any i/o
[0:26] <joshd> see e.g. https://github.com/ceph/ceph/blob/master/qa/workunits/rbd/test_lock_fence.sh and https://github.com/ceph/ceph/blob/master/src/test/librbd/rbdrw.py
[0:28] * baffle (baffle@jump.stenstad.net) has joined #ceph
[0:34] * fred`` (fred@2001:6f8:10c0:0:2010:abec:24d:2500) has joined #ceph
[0:34] * linjan (~linjan@176.195.130.167) Quit (Ping timeout: 480 seconds)
[0:37] * vata (~vata@207.96.182.162) Quit (Quit: Leaving.)
[0:38] * DV (~veillard@2001:41d0:1:d478::1) Quit (Ping timeout: 480 seconds)
[0:38] * Guest55 (~herrsergi@200.77.224.239) Quit (Ping timeout: 480 seconds)
[0:40] * mtb` (~mtb`@157.130.171.46) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[0:40] * jwilkins (~jowilkin@2600:1010:b12c:ef6f:ea2a:eaff:fe08:3f1d) Quit (Ping timeout: 480 seconds)
[0:43] * yguang11 (~yguang11@2001:4998:effd:600:c054:5cdd:9823:962b) Quit ()
[0:46] * samx (~Adium@12.206.204.58) Quit (Quit: Leaving.)
[0:52] <rkeene> I do it slightly different
[0:52] <rkeene> Instead of blacklisting the dead node I just create+flatten a clone and rename it to the image I want to use and rm the old one
[0:53] * bene3 (~bene@nat-pool-bos-t.redhat.com) Quit (Quit: Konversation terminated!)
[0:55] <rkeene> Because there is a time after locking but before using it
[0:55] <rkeene> So it'll be locked with no watchers, but after a few seconds a watcher will come into play
[0:56] <rkeene> Basically... locking is hard and Ceph provides no useful primitives so you have to hedge against corruption
[0:57] * ychen (~ychen@69.25.143.32) Quit ()
[1:04] * moore (~moore@64.202.160.88) Quit (Remote host closed the connection)
[1:05] * stiopa (~stiopa@cpc73828-dals21-2-0-cust630.20-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[1:06] * kawa2014 (~kawa@2001:67c:1560:8007::aac:c1a6) Quit (Ping timeout: 480 seconds)
[1:09] * barra204 (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) has joined #ceph
[1:12] * yuriw (~Adium@2601:645:4380:112c:e977:16d1:11f0:4835) has joined #ceph
[1:16] * kawa2014 (~kawa@2001:67c:1560:8007::aac:c1a6) has joined #ceph
[1:16] * yanzheng (~zhyan@171.216.95.21) has joined #ceph
[1:18] * fsimonce (~simon@host28-31-dynamic.30-79-r.retail.telecomitalia.it) Quit (Quit: Coyote finally caught me)
[1:18] * samx (~Adium@12.206.204.58) has joined #ceph
[1:20] * brian_ (~textual@109.255.114.93) has joined #ceph
[1:21] * brian_ (~textual@109.255.114.93) Quit (Max SendQ exceeded)
[1:21] * brian_ (~textual@109.255.114.93) has joined #ceph
[1:23] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[1:23] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[1:23] * gregmark (~Adium@68.87.42.115) Quit (Quit: Leaving.)
[1:24] * C2J (~c2j@116.226.230.50) has joined #ceph
[1:26] * brian (~textual@109.255.114.93) Quit (Ping timeout: 480 seconds)
[1:29] * rendar (~I@host25-143-dynamic.59-82-r.retail.telecomitalia.it) Quit (Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!)
[1:30] <motk> hrm, ceph-ansible is a bit confusing these days
[1:31] * Salamander_ (~click@158.255.215.203) has joined #ceph
[1:40] * xarses (~xarses@64.124.158.100) Quit (Ping timeout: 480 seconds)
[1:41] <motk> the out-of-the-box vagrant demo just isn't copying keys in place
[1:43] <motk> is seb on chan?
[1:47] * mattbenjamin1 (~mbenjamin@aa2.linuxbox.com) Quit (Quit: Leaving.)
[1:52] * dgurtner (~dgurtner@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[1:54] * EinstCrazy (~EinstCraz@111.30.21.47) has joined #ceph
[1:57] <KaneK> how do I read output of ceph osd tree?
[2:00] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[2:00] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[2:00] * shakamunyi (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) has joined #ceph
[2:01] * Salamander_ (~click@5P6AABQXP.tor-irc.dnsbl.oftc.net) Quit ()
[2:02] * xarses (~xarses@50.141.33.49) has joined #ceph
[2:03] * xarses (~xarses@50.141.33.49) Quit (Remote host closed the connection)
[2:03] * xarses (~xarses@50.141.33.49) has joined #ceph
[2:04] * samx1 (~Adium@12.206.204.58) has joined #ceph
[2:05] * barra204 (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[2:06] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[2:06] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Remote host closed the connection)
[2:10] * sudocat (~dibarra@192.185.1.20) Quit (Ping timeout: 480 seconds)
[2:10] * samx (~Adium@12.206.204.58) Quit (Ping timeout: 480 seconds)
[2:17] * shawniverson (~shawniver@208.38.236.8) has joined #ceph
[2:17] * alram (~alram@cpe-172-250-2-46.socal.res.rr.com) Quit (Quit: leaving)
[2:22] * jdillaman (~jdillaman@pool-108-18-97-82.washdc.fios.verizon.net) has joined #ceph
[2:23] * KaneK (~kane@12.206.204.58) Quit (Quit: KaneK)
[2:23] * xarses (~xarses@50.141.33.49) Quit (Ping timeout: 480 seconds)
[2:24] * samx1 (~Adium@12.206.204.58) Quit (Ping timeout: 480 seconds)
[2:27] * KaZeR (~KaZeR@64.201.252.132) Quit (Remote host closed the connection)
[2:38] * dnovosel (17e91b5a@107.161.19.53) has joined #ceph
[2:39] * garphy is now known as garphy`aw
[2:40] <dnovosel> Just a quick one.. I've seen numerous references [mostly quite old] saying using RBD on an OSD server is a horrible idea. Is this still the case, or has this problem been fixed?
[2:42] * xarses (~xarses@50.141.34.241) has joined #ceph
[2:47] * naoto (~naotok@2401:bd00:b001:8920:27:131:11:254) has joined #ceph
[2:48] * naoto (~naotok@2401:bd00:b001:8920:27:131:11:254) Quit ()
[2:50] * leseb_away is now known as leseb_
[2:50] * leseb_ is now known as leseb_away
[2:51] * leseb_away is now known as leseb_
[2:51] * naoto (~naotok@2401:bd00:b001:8920:27:131:11:254) has joined #ceph
[2:51] * leseb_ is now known as leseb_away
[2:58] * moore (~moore@71-211-73-118.phnx.qwest.net) has joined #ceph
[3:00] * dynamicudpate (~overonthe@199.68.193.62) has joined #ceph
[3:00] * LPG (~LPG@c-50-181-212-148.hsd1.wa.comcast.net) has joined #ceph
[3:01] * leseb_away is now known as leseb_
[3:01] * leseb_ is now known as leseb_away
[3:01] * tomc (~tomc@192.41.52.12) Quit (Quit: tomc)
[3:02] * leseb_away is now known as leseb_
[3:02] * LPG (~LPG@c-50-181-212-148.hsd1.wa.comcast.net) Quit (Remote host closed the connection)
[3:02] * leseb_ is now known as leseb_away
[3:02] * leseb_away is now known as leseb_
[3:03] * mtb` (~mtb`@96-35-37-164.dhcp.stls.mo.charter.com) has joined #ceph
[3:03] * leseb_ is now known as leseb_away
[3:04] * mtb` (~mtb`@96-35-37-164.dhcp.stls.mo.charter.com) Quit ()
[3:04] * tsg (~tgohad@jfdmzpr01-ext.jf.intel.com) Quit (Remote host closed the connection)
[3:06] * dnovosel (17e91b5a@107.161.19.53) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[3:06] * boredatwork (~overonthe@199.68.193.54) Quit (Ping timeout: 480 seconds)
[3:07] * dnovosel (17e91b5a@107.161.19.109) has joined #ceph
[3:10] * leseb_away is now known as leseb_
[3:10] * leseb_ is now known as leseb_away
[3:10] * leseb_away is now known as leseb_
[3:10] * leseb_ is now known as leseb_away
[3:10] * dnovosel (17e91b5a@107.161.19.109) Quit ()
[3:12] * KaZeR (~KaZeR@166.170.43.180) has joined #ceph
[3:12] * dnovosel (17e91b5a@107.161.19.53) has joined #ceph
[3:13] * kefu (~kefu@114.92.110.166) has joined #ceph
[3:14] * KaZeR_ (~KaZeR@2600:1010:b013:cb16:953e:ec92:d58a:9c4f) has joined #ceph
[3:14] * linuxkidd (~linuxkidd@72.sub-70-210-225.myvzw.com) Quit (Ping timeout: 480 seconds)
[3:16] * zhaochao (~zhaochao@60.206.230.18) has joined #ceph
[3:21] * KaZeR (~KaZeR@166.170.43.180) Quit (Ping timeout: 480 seconds)
[3:21] * naoto_ (~naotok@27.131.11.254) has joined #ceph
[3:25] * linuxkidd (~linuxkidd@72.sub-70-210-225.myvzw.com) has joined #ceph
[3:26] * georgem (~Adium@75-119-226-89.dsl.teksavvy.com) has joined #ceph
[3:27] * naoto (~naotok@2401:bd00:b001:8920:27:131:11:254) Quit (Ping timeout: 480 seconds)
[3:32] * KaZeR_ (~KaZeR@2600:1010:b013:cb16:953e:ec92:d58a:9c4f) Quit (Remote host closed the connection)
[3:38] * KaneK (~kane@cpe-76-89-193-31.socal.res.rr.com) has joined #ceph
[3:42] * kefu (~kefu@114.92.110.166) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[3:42] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[3:43] * elder_ (~elder@104.135.14.148) has joined #ceph
[3:43] * moore (~moore@71-211-73-118.phnx.qwest.net) Quit (Remote host closed the connection)
[3:44] * elder (~elder@104.135.14.148) Quit (Read error: Connection reset by peer)
[3:44] * cephalic (~chatzilla@103.224.119.216) has joined #ceph
[3:54] * elder_ (~elder@104.135.14.148) Quit (Quit: Leaving)
[3:56] * shinobu (~oftc-webi@nat-pool-nrt-t1.redhat.com) Quit (Ping timeout: 480 seconds)
[3:56] * t4nk596 (~oftc-webi@192-41-52-12.c7dc.com) Quit (Ping timeout: 480 seconds)
[4:00] * nhm (~nhm@c-50-171-139-246.hsd1.mn.comcast.net) Quit (Quit: Lost terminal)
[4:02] * LeaChim (~LeaChim@host86-185-146-193.range86-185.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[4:07] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[4:23] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[4:24] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[4:27] * dgbaley27 (~matt@75.148.118.217) has joined #ceph
[4:34] * dnovosel (17e91b5a@107.161.19.53) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[4:36] * dnovosel (17e91b5a@107.161.19.53) has joined #ceph
[4:36] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[4:40] * dnovosel (17e91b5a@107.161.19.53) Quit ()
[4:40] * dnovosel (17e91b5a@107.161.19.53) has joined #ceph
[4:41] * leseb_away is now known as leseb_
[4:41] * leseb_ is now known as leseb_away
[4:41] * xarses (~xarses@50.141.34.241) Quit (Ping timeout: 480 seconds)
[4:41] * leseb_away is now known as leseb_
[4:42] * leseb_ is now known as leseb_away
[4:42] * dnovosel (17e91b5a@107.161.19.53) Quit ()
[4:43] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[4:43] * dnovosel (17e91b5a@107.161.19.53) has joined #ceph
[4:45] * moore (~moore@71-211-73-118.phnx.qwest.net) has joined #ceph
[4:46] * moore (~moore@71-211-73-118.phnx.qwest.net) Quit (Remote host closed the connection)
[5:03] * linuxkidd (~linuxkidd@72.sub-70-210-225.myvzw.com) Quit (Quit: Leaving)
[5:06] * kanagaraj (~kanagaraj@121.244.87.117) has joined #ceph
[5:07] * rdas (~rdas@121.244.87.116) has joined #ceph
[5:09] * dnovosel (17e91b5a@107.161.19.53) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[5:10] * Vacuum__ (~Vacuum@88.130.209.253) has joined #ceph
[5:11] * dnovosel (17e91b5a@107.161.19.109) has joined #ceph
[5:13] * amote (~amote@121.244.87.116) has joined #ceph
[5:17] * Vacuum_ (~Vacuum@i59F796F0.versanet.de) Quit (Ping timeout: 480 seconds)
[5:19] * Aal (~Aramande_@se4x.mullvad.net) has joined #ceph
[5:23] * leseb_away is now known as leseb_
[5:23] * leseb_ is now known as leseb_away
[5:31] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[5:40] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[5:44] * swami1 (~swami@49.32.0.225) has joined #ceph
[5:49] * Aal (~Aramande_@4Z9AABPJL.tor-irc.dnsbl.oftc.net) Quit ()
[5:50] * kefu (~kefu@114.92.110.166) has joined #ceph
[5:56] * rraja|afk is now known as rraja
[5:58] * Tumm (~kalmisto@185.12.45.110) has joined #ceph
[5:59] * georgem (~Adium@75-119-226-89.dsl.teksavvy.com) Quit (Quit: Leaving.)
[6:00] * jamespage (~jamespage@2a00:1098:0:80:1000:42:0:1) Quit (Quit: Coyote finally caught me)
[6:00] * jamespage (~jamespage@2a00:1098:0:80:1000:42:0:1) has joined #ceph
[6:14] * kawa2014 (~kawa@2001:67c:1560:8007::aac:c1a6) Quit (Ping timeout: 480 seconds)
[6:15] * jdillaman (~jdillaman@pool-108-18-97-82.washdc.fios.verizon.net) Quit (Quit: jdillaman)
[6:16] * haomaiwang (~haomaiwan@103.15.217.218) has joined #ceph
[6:22] * leseb_away is now known as leseb_
[6:22] * leseb_ is now known as leseb_away
[6:22] * leseb_away is now known as leseb_
[6:22] * leseb_ is now known as leseb_away
[6:23] * leseb_away is now known as leseb_
[6:23] * kefu (~kefu@114.92.110.166) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[6:28] * Tumm (~kalmisto@5P6AABQ2W.tor-irc.dnsbl.oftc.net) Quit ()
[6:29] * leseb_ is now known as leseb_away
[6:31] <geli> whats the best way to disable cephx on my osds durning setup?
[6:32] <motk> whyfor?
[6:34] <geli> the keys are not being distrubuted for some reason using ceph-deploy and this not a production system so the thought is to get it up and running at have a look at the keys later
[6:35] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[6:35] * leseb_away is now known as leseb_
[6:35] * leseb_ is now known as leseb_away
[6:37] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[6:46] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) has joined #ceph
[6:56] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[6:59] * vikhyat (~vumrao@121.244.87.116) has joined #ceph
[7:02] * reed (~reed@75-101-54-18.dsl.static.fusionbroadband.com) Quit (Quit: Ex-Chat)
[7:02] * overclk (~vshankar@121.244.87.117) has joined #ceph
[7:07] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[7:07] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[7:08] * bandrus (~brian@183.sub-70-214-37.myvzw.com) Quit (Quit: Leaving.)
[7:10] * overclk (~vshankar@121.244.87.117) Quit (Ping timeout: 480 seconds)
[7:15] * m0zes (~mozes@beocat.cis.ksu.edu) Quit (Ping timeout: 480 seconds)
[7:19] * m0zes (~mozes@beocat.cis.ksu.edu) has joined #ceph
[7:20] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[7:24] * squ (~Thunderbi@00020d26.user.oftc.net) has joined #ceph
[7:26] * kefu (~kefu@120.253.218.96) has joined #ceph
[7:29] * leseb_away is now known as leseb_
[7:30] * leseb_ is now known as leseb_away
[7:31] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) has joined #ceph
[7:32] * karnan (~karnan@121.244.87.117) has joined #ceph
[7:35] * leseb_away is now known as leseb_
[7:35] * leseb_ is now known as leseb_away
[7:39] * remy1991 (~ravi@115.114.59.182) has joined #ceph
[7:40] * rakeshgm (~rakesh@121.244.87.117) has joined #ceph
[7:53] * Solvius (~OODavo@89.248.169.36) has joined #ceph
[7:58] * dgbaley27 (~matt@75.148.118.217) Quit (Quit: Leaving.)
[8:16] * johnhunter (~hunter@115.27.204.38) has joined #ceph
[8:22] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[8:22] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[8:23] * Solvius (~OODavo@4Z9AABPM0.tor-irc.dnsbl.oftc.net) Quit ()
[8:25] * Be-El (~quassel@fb08-bcf-pc01.computational.bio.uni-giessen.de) has joined #ceph
[8:30] * overclk (~vshankar@121.244.87.117) has joined #ceph
[8:31] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[8:33] <Be-El> hi
[8:51] * bliu (~liub@203.192.156.9) Quit (Ping timeout: 480 seconds)
[8:58] * jasuarez (~jasuarez@38.Red-81-38-77.dynamicIP.rima-tde.net) has joined #ceph
[8:58] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) has joined #ceph
[9:01] * bara (~bara@213.175.37.10) has joined #ceph
[9:02] * dugravot6 (~dugravot6@nat-persul-plg.wifi.univ-lorraine.fr) has joined #ceph
[9:08] * enax (~enax@hq.ezit.hu) has joined #ceph
[9:11] * analbeard (~shw@support.memset.com) has joined #ceph
[9:12] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) Quit (Ping timeout: 480 seconds)
[9:15] * bara (~bara@213.175.37.10) Quit (Remote host closed the connection)
[9:16] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[9:17] * cephalic (~chatzilla@103.224.119.216) Quit (Ping timeout: 480 seconds)
[9:25] * garphy`aw is now known as garphy
[9:32] * natarej_ (natarej@CPE-101-181-159-253.lnse5.cha.bigpond.net.au) Quit (Read error: Connection reset by peer)
[9:32] * natarej_ (natarej@CPE-101-181-159-253.lnse5.cha.bigpond.net.au) has joined #ceph
[9:32] * ksperis (~laurent@46.218.42.103) has joined #ceph
[9:33] * ksperis (~laurent@46.218.42.103) has left #ceph
[9:33] * rendar (~I@host230-177-dynamic.42-79-r.retail.telecomitalia.it) has joined #ceph
[9:35] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[9:37] * fsimonce (~simon@host28-31-dynamic.30-79-r.retail.telecomitalia.it) has joined #ceph
[9:37] * egonzalez (~egonzalez@238.238.14.62.static.jazztel.es) has joined #ceph
[9:38] * b0e (~aledermue@213.95.25.82) has joined #ceph
[9:38] * Bidlo (~oftc-webi@84-244-93-226.static.bluetone.cz) has joined #ceph
[9:42] <Bidlo> Hello guys, I have a problem with installing Ceph on Debian 8 Jessie, can anybody help? To be more concrete, the problem is that I cannot find package named "ceph-deploy" no matter what I do (it's not in official deb repository and neither is available when I follow Preflight tutorial and add the ceph repo). Thanks for any help or tips!
[9:43] * Nicola-1980 (~Nicola-19@2-234-77-205.ip222.fastwebnet.it) has joined #ceph
[9:43] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) has joined #ceph
[9:46] * nils_ (~nils_@doomstreet.collins.kg) has joined #ceph
[9:47] * pabluk_ is now known as pabluk
[9:49] * Kupo1 (~tyler.wil@23.111.254.159) Quit (Read error: Connection reset by peer)
[9:49] * dgurtner (~dgurtner@178.197.231.21) has joined #ceph
[9:49] <zao> Bidlo: According to my logs, people seem to be able to use the packages from places like http://download.ceph.com/debian-hammer/pool/main/c/ceph-deploy/
[9:49] <zao> (the ubuntu ones supposedly install and "work")
[9:50] <zao> Never tried it myself.
[9:51] * daviddcc (~dcasier@80.215.200.122) has joined #ceph
[9:52] * ram_ (~oftc-webi@static-202-65-140-146.pol.net.in) Quit (Ping timeout: 480 seconds)
[9:52] * Kupo1 (~tyler.wil@23.111.254.159) has joined #ceph
[9:52] * branto1 (~branto@ip-78-102-208-28.net.upcbroadband.cz) has joined #ceph
[9:54] <Bidlo> zao: Well, I am able to see all other packages (like ceph, ceph-common, etc) but not ceph-deploy which is strange. There are even mutiple bugs in the bug tracker like this one http://tracker.ceph.com/issues/13544
[9:54] * linjan (~linjan@176.195.130.167) has joined #ceph
[9:54] * ccourtaut_ (~ccourtaut@2001:41d0:1:eed3::1) Quit (Ping timeout: 480 seconds)
[9:56] <boolman> Bidlo: ceph-deploy is not in jessie repo as you can see in http://download.ceph.com/debian-hammer/dists/jessie/main/binary-amd64/Packages
[9:56] <boolman> atleast not in hammer for amd64
[9:57] * johnhunter (~hunter@115.27.204.38) Quit (Ping timeout: 480 seconds)
[9:57] * thomnico (~thomnico@2a01:e35:8b41:120:4560:68c3:4199:ddb7) has joined #ceph
[10:02] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[10:04] * benner_ (~benner@188.166.111.206) has joined #ceph
[10:05] * linjan (~linjan@176.195.130.167) Quit (Ping timeout: 480 seconds)
[10:05] * benner (~benner@188.166.111.206) Quit (Read error: Connection reset by peer)
[10:05] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[10:05] * overclk (~vshankar@121.244.87.117) Quit (Ping timeout: 480 seconds)
[10:05] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Ping timeout: 480 seconds)
[10:06] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[10:07] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[10:07] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[10:09] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[10:09] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[10:10] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[10:10] * Bidlo (~oftc-webi@84-244-93-226.static.bluetone.cz) Quit (Ping timeout: 480 seconds)
[10:10] * daviddcc (~dcasier@80.215.200.122) Quit (Read error: Connection reset by peer)
[10:12] * rotbeard (~redbeard@185.32.80.238) has joined #ceph
[10:15] * chasmo77 (~chas77@158.183-62-69.ftth.swbr.surewest.net) has joined #ceph
[10:16] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[10:17] * linjan (~linjan@176.195.130.167) has joined #ceph
[10:17] * bara (~bara@nat-pool-brq-u.redhat.com) has joined #ceph
[10:25] * derjohn_mob (~aj@fw.gkh-setu.de) has joined #ceph
[10:27] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[10:34] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[10:34] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[10:37] * rdas (~rdas@121.244.87.116) has joined #ceph
[10:38] * kefu (~kefu@120.253.218.96) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[10:40] * odi (~quassel@2a00:12c0:1015:136::9) has joined #ceph
[10:45] * bitserker (~toni@81.184.9.72.dyn.user.ono.com) has joined #ceph
[10:47] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[10:49] * kefu (~kefu@114.92.110.166) has joined #ceph
[10:50] * nardial (~ls@dslb-084-063-136-171.084.063.pools.vodafone-ip.de) has joined #ceph
[10:53] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[10:57] * odi (~quassel@2a00:12c0:1015:136::9) Quit (Remote host closed the connection)
[10:57] * bara (~bara@nat-pool-brq-u.redhat.com) Quit (Ping timeout: 480 seconds)
[10:57] * odi (~quassel@2a00:12c0:1015:136::9) has joined #ceph
[11:00] * kosmas (~kosmasgia@147.52.131.4) has joined #ceph
[11:01] * LeaChim (~LeaChim@host86-185-146-193.range86-185.btcentralplus.com) has joined #ceph
[11:02] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[11:07] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[11:07] * rakeshgm (~rakesh@121.244.87.117) Quit (Quit: Leaving)
[11:08] * rakeshgm (~rakesh@121.244.87.117) has joined #ceph
[11:08] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[11:09] * remy1991 (~ravi@115.114.59.182) Quit (Ping timeout: 480 seconds)
[11:10] * thomnico (~thomnico@2a01:e35:8b41:120:4560:68c3:4199:ddb7) Quit (Quit: Ex-Chat)
[11:14] * kosmas (~kosmasgia@147.52.131.4) has left #ceph
[11:16] * nardial (~ls@dslb-084-063-136-171.084.063.pools.vodafone-ip.de) Quit (Quit: Leaving)
[11:19] * Silentspy (~N3X15@94.103.175.86) has joined #ceph
[11:32] * odi_ (~quassel@farce.de) has joined #ceph
[11:33] * cephalic (~chatzilla@103.224.119.216) has joined #ceph
[11:33] * olid1118 (~olid1982@aftr-185-17-206-128.dynamic.mnet-online.de) has joined #ceph
[11:33] <john341> Using libvirt/librados to access rbd for VMs, what protocol is used? I'm not sure how the block devices are connected accessed through the compute end, it doesn't seem to be iSCSI, is it a rbd protocol that manages this?
[11:34] <boolman> what is the required auth capabilites for a cephfs user?
[11:35] <boolman> rwx on the data and metadata pools, r on mons, rwx on mds ?
[11:35] <Kvisle> john341: it uses rados
[11:35] * odi (~quassel@2a00:12c0:1015:136::9) Quit (Ping timeout: 480 seconds)
[11:35] <Kvisle> john341: well, rbd, http://docs.ceph.com/docs/v0.94/rbd/qemu-rbd/
[11:36] * kefu (~kefu@114.92.110.166) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[11:38] * haomaiwang (~haomaiwan@103.15.217.218) Quit (Remote host closed the connection)
[11:40] <john341> Kvisle: thanks for sharing. I'm not sure if one gotta read the source to understand how it works. Do you know how it works, will a secure tcp-connection be established to the cluster or how does the access work more in detail?
[11:41] <lathiat> john341: the rbd client is a ceph client, it speaks to the entire ceph clustre like any ceph client would, its essentially part of the cluster, as a client
[11:41] <lathiat> so it speaks directly with OSDs
[11:41] <lathiat> with multiple connections
[11:42] <lathiat> its a rados client, maybe more helpful
[11:42] * rdas (~rdas@121.244.87.116) Quit (Ping timeout: 480 seconds)
[11:42] <Kvisle> john341: the client needs a key to authenticate, with access to perform io on the pools with the block devices in them .. + what lathiat said
[11:44] <john341> okay, so in a nutshell it seems to be a radois client and tcp connections
[11:44] <Kvisle> yes.
[11:45] <john341> I guess the key is only for autentication, but the data transfer isnt encrypted and should on an isolated network or secure tunnel?
[11:47] <lathiat> general best practice is that the ceph client-osd network is separate
[11:47] <lathiat> i would suggest looking at some of the talks on yhoutube about ceph it explains most of this with pretty diagrams, etc and goes over it in more detail
[11:49] * Silentspy (~N3X15@6YRAAA8W8.tor-irc.dnsbl.oftc.net) Quit ()
[11:50] * alrick (~alrick@46.39.53.53) has joined #ceph
[11:52] * rdas (~rdas@121.244.87.116) has joined #ceph
[11:56] <john341> lathiat: yerps, will do. thanks!
[11:57] * shawniverson (~shawniver@208.38.236.8) Quit (Remote host closed the connection)
[12:00] <Kvisle> is that recommendation based on anything other than network congestion?
[12:01] <lathiat> Kvisle: that is the general basis for it, in any realistically sized cluster you dont want to cut your bandwidth down by double transmitting over the same interfaces. tripply important for gigE
[12:01] <lathiat> but also latency will suffer if the link starts getting congested etc
[12:01] * overclk (~vshankar@121.244.87.117) has joined #ceph
[12:02] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[12:02] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[12:02] <Kvisle> just checking that I've understood it correctly. our cluster is currently not designed in that way, but we're far from that kind of throughput
[12:02] <Kvisle> will eventually change, but we'll adapt when needed.
[12:02] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[12:03] * overclk (~vshankar@121.244.87.117) Quit ()
[12:04] * kefu (~kefu@114.92.110.166) has joined #ceph
[12:06] * olid1118 (~olid1982@aftr-185-17-206-128.dynamic.mnet-online.de) Quit (Ping timeout: 480 seconds)
[12:06] <lathiat> Kvisle: are you GE or 10GE?
[12:06] <lathiat> you suffer fast from it on GE, you can probably get away with it on 10GE
[12:13] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[12:13] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[12:24] * zhaochao (~zhaochao@60.206.230.18) Quit (Quit: ChatZilla 0.9.92 [Iceweasel 38.4.0/20151103222033])
[12:26] * alrick (~alrick@46.39.53.53) Quit (Remote host closed the connection)
[12:26] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[12:26] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[12:28] * naoto_ (~naotok@27.131.11.254) Quit (Quit: Leaving...)
[12:39] * karnan (~karnan@121.244.87.117) Quit (Remote host closed the connection)
[12:48] * bene2 (~bene@2601:18c:8300:f3ae:ea2a:eaff:fe08:3c7a) has joined #ceph
[12:50] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[12:50] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[12:55] * kefu (~kefu@114.92.110.166) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[12:56] * steveeJ (~junky@HSI-KBW-134-3-203-161.hsi14.kabel-badenwuerttemberg.de) Quit (Remote host closed the connection)
[12:57] * Bidlo (~oftc-webi@84-244-93-226.static.bluetone.cz) has joined #ceph
[12:57] * kefu (~kefu@114.92.110.166) has joined #ceph
[12:58] * enax (~enax@hq.ezit.hu) Quit (Remote host closed the connection)
[13:00] * enax (~enax@hq.ezit.hu) has joined #ceph
[13:03] * kanagaraj (~kanagaraj@121.244.87.117) Quit (Quit: Leaving)
[13:04] <Kurt> hi
[13:04] <Kurt> weisst du, ob wir/ich am 18:11 irgendwas am ceph cluster gemacht haben?
[13:04] <Kurt> ca. um die mittagszeit
[13:05] <Kurt> sorry
[13:05] <Kurt> wrong window ;-)
[13:06] * rakeshgm (~rakesh@121.244.87.117) Quit (Remote host closed the connection)
[13:07] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Ping timeout: 480 seconds)
[13:17] * bara (~bara@nat-pool-brq-u.redhat.com) has joined #ceph
[13:17] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Ping timeout: 480 seconds)
[13:20] * dugravot6 (~dugravot6@nat-persul-plg.wifi.univ-lorraine.fr) Quit (Quit: Leaving.)
[13:25] * C2J_ (~c2j@116.226.230.50) has joined #ceph
[13:32] * C2J (~c2j@116.226.230.50) Quit (Ping timeout: 480 seconds)
[13:33] * C2J_ (~c2j@116.226.230.50) Quit (Remote host closed the connection)
[13:38] <boolman> is it possible to restrict clients to a specific cephfs namespace?
[13:39] * DV (~veillard@2001:41d0:1:d478::1) has joined #ceph
[13:40] * bara (~bara@nat-pool-brq-u.redhat.com) Quit (Ping timeout: 480 seconds)
[13:42] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[13:42] * jdillaman (~jdillaman@pool-108-18-97-82.washdc.fios.verizon.net) has joined #ceph
[13:43] * odi_ (~quassel@farce.de) Quit (Remote host closed the connection)
[13:43] * haomaiwang (~haomaiwan@103.15.217.218) has joined #ceph
[13:44] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[13:46] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[13:46] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[13:50] * bara (~bara@213.175.37.10) has joined #ceph
[13:51] * cephalic (~chatzilla@103.224.119.216) Quit (Ping timeout: 480 seconds)
[13:51] * Icey (~Icey@0001bbad.user.oftc.net) Quit (Remote host closed the connection)
[13:51] * Icey (~Icey@pool-74-109-7-163.phlapa.fios.verizon.net) has joined #ceph
[13:52] * wjw-freebsd (~wjw@vpn.ecoracks.nl) has joined #ceph
[13:55] * EinstCra_ (~EinstCraz@111.30.21.47) has joined #ceph
[13:58] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[13:58] * odi (~quassel@2a00:12c0:1015:136::9) has joined #ceph
[14:01] * ira (~ira@c-73-238-173-100.hsd1.ma.comcast.net) has joined #ceph
[14:02] * EinstCrazy (~EinstCraz@111.30.21.47) Quit (Ping timeout: 480 seconds)
[14:08] * odi (~quassel@2a00:12c0:1015:136::9) Quit (Remote host closed the connection)
[14:09] * kefu (~kefu@114.92.110.166) Quit (Max SendQ exceeded)
[14:09] * georgem (~Adium@184.151.190.145) has joined #ceph
[14:10] * kefu (~kefu@114.92.110.166) has joined #ceph
[14:12] * thomnico (~thomnico@2a01:e34:edf7:4710:84aa:2e97:b789:9f66) has joined #ceph
[14:14] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Ping timeout: 480 seconds)
[14:19] * swami1 (~swami@49.32.0.225) Quit (Quit: Leaving.)
[14:19] * kefu (~kefu@114.92.110.166) Quit (Max SendQ exceeded)
[14:20] * kefu (~kefu@114.92.110.166) has joined #ceph
[14:21] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[14:29] * dneary (~dneary@175.100.206.155) has joined #ceph
[14:33] * georgem (~Adium@184.151.190.145) Quit (Quit: Leaving.)
[14:33] * EinstCra_ (~EinstCraz@111.30.21.47) Quit (Remote host closed the connection)
[14:35] * remy1991 (~ravi@115.114.59.182) has joined #ceph
[14:38] <Heebie> Does anyone know what happens if you shut down an entire ceph setup, re-rack it, and turn it all back on? (It's not in-use by any clients etc.. at the moment.)
[14:38] <Heebie> Also, does anyone know if you can change to using a cluster network + a client network, from just having a single network, on-the-fly?
[14:40] * haomaiwang (~haomaiwan@103.15.217.218) Quit (Remote host closed the connection)
[14:41] * haomaiwang (~haomaiwan@103.15.217.218) has joined #ceph
[14:49] * haomaiwang (~haomaiwan@103.15.217.218) Quit (Ping timeout: 480 seconds)
[14:49] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[14:49] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[14:49] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[14:52] * georgem (~Adium@206.108.127.16) has joined #ceph
[14:53] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[14:54] * tsg (~tgohad@jfdmzpr04-ext.jf.intel.com) has joined #ceph
[14:56] * nini123 (~jiji123@213.205.251.197) has joined #ceph
[14:56] * thomnico (~thomnico@2a01:e34:edf7:4710:84aa:2e97:b789:9f66) Quit (Quit: Ex-Chat)
[14:56] * olid1118 (~olid1982@193.24.209.54) has joined #ceph
[15:01] * nono123 (~jiji123@103.16.26.71) has joined #ceph
[15:01] * EinstCrazy (~EinstCraz@117.13.201.130) has joined #ceph
[15:02] * bene2 (~bene@2601:18c:8300:f3ae:ea2a:eaff:fe08:3c7a) Quit (Quit: Konversation terminated!)
[15:02] * vikhyat (~vumrao@121.244.87.116) Quit (Quit: Leaving)
[15:03] * Wielebny (~Icedove@cl-927.waw-01.pl.sixxs.net) Quit (Quit: Wielebny)
[15:08] * nini123 (~jiji123@213.205.251.197) Quit (Ping timeout: 480 seconds)
[15:14] * tsg (~tgohad@jfdmzpr04-ext.jf.intel.com) Quit (Remote host closed the connection)
[15:14] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[15:15] <s3an2_> Heebie, shutdown and power up worked well for me - I just set noout before doing it
[15:19] * blank (~ghostnote@6YRAAA854.tor-irc.dnsbl.oftc.net) has joined #ceph
[15:19] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[15:23] * yanzheng (~zhyan@171.216.95.21) Quit (Quit: This computer has gone to sleep)
[15:24] * harold (~hamiller@71-94-227-123.dhcp.mdfd.or.charter.com) has joined #ceph
[15:24] * yanzheng (~zhyan@171.216.95.21) has joined #ceph
[15:26] * Kurt (~Adium@2001:628:1:5:b56b:1f38:c891:bb94) Quit (Quit: Leaving.)
[15:28] * Bidlo (~oftc-webi@84-244-93-226.static.bluetone.cz) Quit (Ping timeout: 480 seconds)
[15:28] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[15:28] * brad_mssw (~brad@66.129.88.50) has joined #ceph
[15:28] <mfa298> I've turned off a small test cluster and it was all fine after. The only time I can see there being real issues is it you have to re-ip at the same time.
[15:28] * harold (~hamiller@71-94-227-123.dhcp.mdfd.or.charter.com) Quit ()
[15:29] <mfa298> noin / noup might be useful for booting it up as well. Turn on mons, set noup/noin, turn on storage, let it sit for a bit then unset.
[15:29] * bara (~bara@213.175.37.10) Quit (Quit: Bye guys! (??????????????????? ?????????)
[15:29] <mfa298> I've done similar when rebooting osd nodes with lots of OSDs per node as it results in fewer OSD map updates
[15:31] * bara (~bara@213.175.37.10) has joined #ceph
[15:33] * krypto (~krypto@103.252.26.68) has joined #ceph
[15:33] * sage (~quassel@2607:f298:6050:709d:38fb:6fab:296b:36e4) Quit (Quit: No Ping reply in 180 seconds.)
[15:34] * remy1991 (~ravi@115.114.59.182) Quit (Ping timeout: 480 seconds)
[15:35] * sage (~quassel@2607:f298:6050:709d:895:729e:bdb:ab84) has joined #ceph
[15:35] * ChanServ sets mode +o sage
[15:35] * kawa2014 (~kawa@12.228.154.70) has joined #ceph
[15:35] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[15:37] * derjohn_mob (~aj@fw.gkh-setu.de) Quit (Ping timeout: 480 seconds)
[15:39] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[15:39] * olid1118 (~olid1982@193.24.209.54) Quit (Ping timeout: 480 seconds)
[15:40] <Heebie> The largest nodes we have in this cluster is 3 OSD's, most are just 1 OSD... but there are 15 servers. Thanks for the info s3an2_ and mfa298
[15:41] * yanzheng (~zhyan@171.216.95.21) Quit (Quit: This computer has gone to sleep)
[15:46] <Luftwaffels> open question, deploying/managing ceph with puppet, what's everyones feeling there?
[15:47] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[15:47] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Remote host closed the connection)
[15:47] * derjohn_mob (~aj@fw.gkh-setu.de) has joined #ceph
[15:49] * blank (~ghostnote@6YRAAA854.tor-irc.dnsbl.oftc.net) Quit ()
[15:49] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[15:49] * vbellur (~vijay@c-71-234-227-202.hsd1.ma.comcast.net) Quit (Ping timeout: 480 seconds)
[15:50] <monsted> ooh, do want: http://www.anandtech.com/show/9830/western-digital-expands-hgst-helium-drive-lineup-with-10tb-ultrastar-he10
[15:52] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[15:52] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[15:55] * cephalic (~chatzilla@103.224.119.216) has joined #ceph
[15:57] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Read error: Connection reset by peer)
[15:57] * jasuarez (~jasuarez@38.Red-81-38-77.dynamicIP.rima-tde.net) Quit (Ping timeout: 480 seconds)
[15:58] * elder (~elder@12.199.200.124) has joined #ceph
[15:58] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[15:59] * mattbenjamin1 (~mbenjamin@76-206-42-105.lightspeed.livnmi.sbcglobal.net) has joined #ceph
[16:00] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Read error: Connection reset by peer)
[16:00] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[16:00] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Read error: Connection reset by peer)
[16:01] * newdave_ (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[16:06] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[16:07] * thomnico (~thomnico@2a01:e35:8b41:120:4560:68c3:4199:ddb7) has joined #ceph
[16:09] <Tetard> whee
[16:09] <Tetard> 10 stk,tak
[16:13] * vbellur (~vijay@nat-pool-bos-u.redhat.com) has joined #ceph
[16:22] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[16:22] * rraja is now known as rraja|afk
[16:23] * ibravo (~ibravo@72.83.69.64) has joined #ceph
[16:25] * mattbenjamin1 (~mbenjamin@76-206-42-105.lightspeed.livnmi.sbcglobal.net) Quit (Ping timeout: 480 seconds)
[16:26] * mtb` (~mtb`@157.130.171.46) has joined #ceph
[16:28] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[16:29] * enax (~enax@hq.ezit.hu) Quit (Ping timeout: 480 seconds)
[16:31] * mancdaz (~mancdaz@2a00:1a48:7806:117:be76:4eff:fe08:7623) Quit (Remote host closed the connection)
[16:31] * kefu is now known as kefu|afk
[16:31] * kefu|afk is now known as kefu
[16:32] * mancdaz (~mancdaz@2a00:1a48:7806:117:be76:4eff:fe08:7623) has joined #ceph
[16:33] * lcurtis_ (~lcurtis@47.19.105.250) has joined #ceph
[16:34] * bara (~bara@213.175.37.10) Quit (Ping timeout: 480 seconds)
[16:35] * kefu is now known as kefu|afk
[16:35] * kefu|afk is now known as kefu
[16:37] * xarses (~xarses@64.124.158.100) has joined #ceph
[16:37] * ircolle (~ircolle@nat-pool-bos-u.redhat.com) has joined #ceph
[16:39] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Remote host closed the connection)
[16:41] * lcurtis_ (~lcurtis@47.19.105.250) Quit (Remote host closed the connection)
[16:41] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) Quit (Read error: Connection reset by peer)
[16:41] * rotbeard (~redbeard@185.32.80.238) Quit (Quit: Leaving)
[16:43] * kefu is now known as kefu|afk
[16:43] * lcurtis_ (~lcurtis@47.19.105.250) has joined #ceph
[16:46] * bara (~bara@nat-pool-brq-u.redhat.com) has joined #ceph
[16:47] * ircolle (~ircolle@nat-pool-bos-u.redhat.com) has left #ceph
[16:48] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[16:48] * EinstCrazy (~EinstCraz@117.13.201.130) Quit (Remote host closed the connection)
[16:49] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[16:50] * ircolle (~ircolle@nat-pool-bos-u.redhat.com) has joined #ceph
[16:51] * nono123 (~jiji123@103.16.26.71) Quit (Ping timeout: 480 seconds)
[16:51] * haomaiwang (~haomaiwan@103.15.217.218) has joined #ceph
[16:52] * nono123 (~jiji123@213.205.251.197) has joined #ceph
[16:53] * nini123 (~jiji123@103.16.26.71) has joined #ceph
[16:55] * rikai1 (~AluAlu@nl12x.mullvad.net) has joined #ceph
[16:56] * kefu|afk (~kefu@114.92.110.166) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[16:56] * gregmark (~Adium@68.87.42.115) has joined #ceph
[16:58] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[16:58] * olid1118 (~olid1982@aftr-185-17-206-128.dynamic.mnet-online.de) has joined #ceph
[16:58] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) has joined #ceph
[16:59] * joshd1 (~jdurgin@68-119-140-18.dhcp.ahvl.nc.charter.com) has joined #ceph
[17:00] * haomaiwang (~haomaiwan@103.15.217.218) Quit (Remote host closed the connection)
[17:00] * nono123 (~jiji123@213.205.251.197) Quit (Ping timeout: 480 seconds)
[17:01] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:02] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:02] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:03] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:03] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:04] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:04] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:04] * kefu (~kefu@114.92.110.166) has joined #ceph
[17:05] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:05] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:05] * sudocat (~dibarra@2602:306:8bc7:4c50::46) has joined #ceph
[17:06] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:06] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:06] * dneary (~dneary@175.100.206.155) Quit (Ping timeout: 480 seconds)
[17:07] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:07] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[17:07] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[17:07] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:08] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:08] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:09] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:09] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:10] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:10] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:11] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:11] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:12] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:12] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:13] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:13] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) has joined #ceph
[17:13] * haomaiwang (~haomaiwan@60-250-10-240.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[17:16] * dnovosel (17e91b5a@107.161.19.109) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[17:16] * newdave_ (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Read error: No route to host)
[17:16] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[17:17] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) Quit (Quit: Leaving)
[17:17] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) has joined #ceph
[17:18] * KaneK (~kane@cpe-76-89-193-31.socal.res.rr.com) Quit (Quit: KaneK)
[17:18] * fretb (~fretb@n-3n.static-37-72-162.as30961.net) Quit (Quit: leaving)
[17:18] * fretb (~fretb@n-3n.static-37-72-162.as30961.net) has joined #ceph
[17:22] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[17:25] * rikai1 (~AluAlu@nl12x.mullvad.net) Quit ()
[17:25] * johnavp1989 (~jpetrini@8.39.115.8) has joined #ceph
[17:34] * moore (~moore@64.202.160.88) has joined #ceph
[17:34] * moore (~moore@64.202.160.88) Quit (Remote host closed the connection)
[17:35] * moore (~moore@64.202.160.88) has joined #ceph
[17:35] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) has joined #ceph
[17:37] * pabluk is now known as pabluk_
[17:38] * samx (~Adium@12.206.204.58) has joined #ceph
[17:40] * mattbenjamin1 (~mbenjamin@aa2.linuxbox.com) has joined #ceph
[17:43] * joshd1 (~jdurgin@68-119-140-18.dhcp.ahvl.nc.charter.com) Quit (Quit: Leaving.)
[17:44] * mtb` (~mtb`@157.130.171.46) Quit (Read error: Connection reset by peer)
[17:44] * nini123 (~jiji123@103.16.26.71) Quit (Ping timeout: 480 seconds)
[17:44] * cfreak200 (~cfreak200@185.104.140.172) Quit (Remote host closed the connection)
[17:52] * Bwana (~isaxi@94.242.228.108) has joined #ceph
[17:52] * b0e (~aledermue@213.95.25.82) Quit (Quit: Leaving.)
[17:52] <MACscr> 10TB is a lot of a data to resync when that drive fails in an array
[17:54] * EinstCrazy (~EinstCraz@117.13.201.130) has joined #ceph
[17:55] * brutuscat (~brutuscat@123.Red-83-55-155.dynamicIP.rima-tde.net) Quit (Remote host closed the connection)
[17:57] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[17:57] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[17:58] * derjohn_mob (~aj@fw.gkh-setu.de) Quit (Ping timeout: 480 seconds)
[17:59] * kefu (~kefu@114.92.110.166) Quit (Max SendQ exceeded)
[18:00] * kefu (~kefu@114.92.110.166) has joined #ceph
[18:00] * bandrus (~brian@143.sub-70-211-67.myvzw.com) has joined #ceph
[18:01] * branto1 (~branto@ip-78-102-208-28.net.upcbroadband.cz) Quit (Ping timeout: 480 seconds)
[18:02] * EinstCrazy (~EinstCraz@117.13.201.130) Quit (Ping timeout: 480 seconds)
[18:04] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[18:05] * ircolle (~ircolle@nat-pool-bos-u.redhat.com) Quit (Ping timeout: 480 seconds)
[18:08] * kefu (~kefu@114.92.110.166) Quit (Ping timeout: 480 seconds)
[18:08] * LPG (~LPG@c-50-181-212-148.hsd1.wa.comcast.net) has joined #ceph
[18:13] * elder (~elder@12.199.200.124) Quit (Quit: Leaving)
[18:16] * zaitcev (~zaitcev@c-50-130-189-82.hsd1.nm.comcast.net) has joined #ceph
[18:16] * nils_ (~nils_@doomstreet.collins.kg) Quit (Quit: This computer has gone to sleep)
[18:18] <mfa298> to some extent that depends on how much data you have in your cluster. For some of the big users 10T isn't much data
[18:22] * Bwana (~isaxi@5P6AABRJL.tor-irc.dnsbl.oftc.net) Quit ()
[18:28] * bara (~bara@nat-pool-brq-u.redhat.com) Quit (Ping timeout: 480 seconds)
[18:35] * derjohn_mob (~aj@88.128.81.9) has joined #ceph
[18:35] <IcePic> or if hundreds of nodes are helping out with those 10T
[18:36] * egonzalez (~egonzalez@238.238.14.62.static.jazztel.es) Quit (Quit: Saliendo)
[18:38] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[18:42] * mykola (~Mikolaj@91.225.201.107) has joined #ceph
[18:43] * tsg (~tgohad@134.134.139.72) has joined #ceph
[18:45] * tsg_ (~tgohad@jfdmzpr01-ext.jf.intel.com) has joined #ceph
[18:45] * elder (~elder@104.135.14.148) has joined #ceph
[18:48] <lookcrabs> is ceph df base 2 or base 10?
[18:50] * tsg (~tgohad@134.134.139.72) Quit (Remote host closed the connection)
[18:53] * pabluk_ (~pabluk@2a01:e34:ec16:93e0:982a:8a61:4172:cb94) Quit (Ping timeout: 480 seconds)
[18:53] * TMM (~hp@178-84-46-106.dynamic.upc.nl) has joined #ceph
[18:55] <Kvisle> lookcrabs: do you mean to ask base 1000 or base 1024?
[18:55] <Kvisle> if not, I don't understand the question - it's not outputing binary, no.
[18:56] <Kvisle> the numbers I get are base 1024
[18:57] <Kvisle> (ceph df -f json-pretty will give you full numbers)
[18:57] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[18:57] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Remote host closed the connection)
[19:00] * bitserker (~toni@81.184.9.72.dyn.user.ono.com) Quit (Quit: Leaving.)
[19:01] * pabluk_ (~pabluk@2a01:e34:ec16:93e0:fdf0:1aa5:37c5:afb7) has joined #ceph
[19:03] * samx (~Adium@12.206.204.58) has left #ceph
[19:03] * reed (~reed@216.38.134.18) has joined #ceph
[19:04] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) Quit (Quit: Leaving)
[19:07] * ibravo2 (~ibravo@72.83.69.64) has joined #ceph
[19:07] * ibravo (~ibravo@72.83.69.64) Quit (Read error: Connection reset by peer)
[19:08] * krypto (~krypto@103.252.26.68) Quit (Ping timeout: 480 seconds)
[19:09] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[19:09] * krypto (~krypto@G68-121-13-85.sbcis.sbc.com) has joined #ceph
[19:13] * evilrob (~evilrob@2600:3c00::f03c:91ff:fedf:1d3d) Quit (Ping timeout: 480 seconds)
[19:18] * TMM (~hp@178-84-46-106.dynamic.upc.nl) Quit (Remote host closed the connection)
[19:20] <lookcrabs> thanks Kvisle !! wasn't sure
[19:21] <lookcrabs> IE is it TB or TiB was what I was unsure about
[19:21] <lookcrabs> so it is TiB just to confirm?
[19:22] <Kvisle> TiB
[19:22] * TMM (~hp@178-84-46-106.dynamic.upc.nl) has joined #ceph
[19:23] <lookcrabs> I always thought it was TB. I've been converting TB to TiB for the time being as I am seeing weird differences between the radosgw quota setting
[19:23] <Kvisle> radosgw measures higher usage than what you find in your pools
[19:23] <Kvisle> at least in my setup
[19:23] <Kvisle> I don't know why that is
[19:24] * onlyanegg (~tcouto@12.252.243.114) has joined #ceph
[19:25] <Kvisle> in my case it's about 25%
[19:25] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[19:26] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[19:26] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[19:27] * remy1991 (~ravi@122.170.40.245) has joined #ceph
[19:36] * Be-El (~quassel@fb08-bcf-pc01.computational.bio.uni-giessen.de) Quit (Remote host closed the connection)
[19:36] <georgem> anybody familiar with supermicro servers? I'm trying to see what's the difference between http://www.supermicro.com/products/system/4U/6048/SSG-6048R-E1CR36H.cfm and http://www.supermicro.com/products/system/4U/6048/SSG-6048R-E1CR36L.cfm (http://www.supermicro.com/products/nfo/storage.cfm)
[19:38] * Nephyrin (~dontron@89.248.171.134) has joined #ceph
[19:38] <mfa298> looking at both pages they have different LSI HBAs and possibly some other small differences
[19:39] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[19:40] <mfa298> although in reality the difference between the two HBA's might just be firmware. Some of the HBAs can take IR (Raid) or IT (JBOD) firmware
[19:40] <georgem> yes, one has LSI 3008 and the other one has LSI 3108 AOC
[19:41] <mfa298> Unless you're going for lots of storage (i.e. lots of those boxes) you might be better with something smaller with ceph
[19:43] <georgem> I am going for a lot of those boxes
[19:43] <mfa298> we did also have issues getting Linux to install on something similar, Although we didn't have the two SSD drives in the back so the OS was on a couple of the 3.5" bays, we found it didn't always try searching the same drives to boot from. I suspect with with the SSDs that issue would disappear
[19:43] <georgem> yes, we have a pair of ssd in the back drive bay
[19:44] <mfa298> what sort of size cluster are you going for ?
[19:44] <georgem> ~10 PB
[19:44] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[19:45] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Quit: Bye guys! (??????????????????? ?????????)
[19:45] * derjohn_mob (~aj@88.128.81.9) Quit (Ping timeout: 480 seconds)
[19:45] <mfa298> you'll probably want to spec the CPU and ram up quite high on them, but that's a decent sized cluster
[19:45] <mfa298> what drives are you looking to use.
[19:46] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[19:50] * wjw-freebsd (~wjw@vpn.ecoracks.nl) Quit (Ping timeout: 480 seconds)
[19:50] * stiopa (~stiopa@cpc73828-dals21-2-0-cust630.20-2.cable.virginm.net) has joined #ceph
[19:51] <georgem> we have 4 TB and 6 TB SAS 12 Gb/s 7.2K
[19:53] <mfa298> sounds like it could be a nice cluster (we're around 15PB but using SMR drives which can be very slow)
[19:57] <georgem> yes, we also don't have SSD-based journals because our use case is large S3 files
[20:06] * ircolle (~ircolle@nat-pool-bos-u.redhat.com) has joined #ceph
[20:08] * alram (~alram@cpe-172-250-2-46.socal.res.rr.com) has joined #ceph
[20:08] * Nephyrin (~dontron@89.248.171.134) Quit ()
[20:09] <T1> keep in mind, that the LSI controllers in newish Dell servers cannot be flashed to IT firmware - you'll brick the entire server
[20:09] <T1> I tried to reflash a PERC H310 to IT..
[20:10] <T1> I couldn't get past POST since there was an unknown storage controller in the internal pcie storage slot (which is NO way near default PCIe)
[20:11] <T1> I had to get a replacement from dell
[20:19] * remy1991 (~ravi@122.170.40.245) Quit (Ping timeout: 480 seconds)
[20:19] * krypto (~krypto@G68-121-13-85.sbcis.sbc.com) Quit (Read error: Connection reset by peer)
[20:20] * krypto (~krypto@103.252.26.68) has joined #ceph
[20:22] * dgurtner (~dgurtner@178.197.231.21) Quit (Ping timeout: 480 seconds)
[20:28] * linjan (~linjan@176.195.130.167) Quit (Ping timeout: 480 seconds)
[20:30] * vata (~vata@cable-21.246.173-197.electronicbox.net) has joined #ceph
[20:31] * jwandborg (~Lattyware@89.248.169.36) has joined #ceph
[20:41] * mykola (~Mikolaj@91.225.201.107) Quit (Quit: away)
[20:41] * brutuscat (~brutuscat@105.34.133.37.dynamic.jazztel.es) has joined #ceph
[20:43] * krypto (~krypto@103.252.26.68) Quit (Quit: Leaving)
[20:44] * linuxkidd (~linuxkidd@72.sub-70-210-225.myvzw.com) has joined #ceph
[20:52] * dnovosel (87f5300c@107.161.19.109) has joined #ceph
[20:54] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[20:54] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[21:01] * jwandborg (~Lattyware@6YRAAA9LF.tor-irc.dnsbl.oftc.net) Quit ()
[21:01] * owasserm_ (~owasserm@a212-238-239-152.adsl.xs4all.nl) has joined #ceph
[21:02] * owasserm_ (~owasserm@a212-238-239-152.adsl.xs4all.nl) Quit (Remote host closed the connection)
[21:02] * mtb` (~mtb`@157.130.171.46) has joined #ceph
[21:08] * KaneK (~kane@cpe-76-89-193-31.socal.res.rr.com) has joined #ceph
[21:17] * thomnico (~thomnico@2a01:e35:8b41:120:4560:68c3:4199:ddb7) Quit (Ping timeout: 480 seconds)
[21:17] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[21:18] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) has joined #ceph
[21:19] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[21:21] * reed (~reed@216.38.134.18) Quit (Ping timeout: 480 seconds)
[21:24] * brutuscat (~brutuscat@105.34.133.37.dynamic.jazztel.es) Quit (Remote host closed the connection)
[21:29] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) has joined #ceph
[21:30] * KaZeR (~KaZeR@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[21:30] * cephalic (~chatzilla@103.224.119.216) Quit (Ping timeout: 480 seconds)
[21:31] * reed (~reed@216.38.134.18) has joined #ceph
[21:37] * rapedex (~WedTM@94.242.228.108) has joined #ceph
[21:39] <lookcrabs> mfa298: may I ask what your setup is? IE what is your typical storage server setup?
[21:39] * rendar (~I@host230-177-dynamic.42-79-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[21:39] * ngoswami (~ngoswami@1.39.17.122) has joined #ceph
[21:40] <T1> mmm 10T non-shingle drives
[21:41] <mfa298> lookcrabs: 2U Supermicro box with something near the best CPU possible (2x 12c+HT), 256GB ram, and around 70 drives in JBODs on each - It's not a setup I'd recommend (we're very much after lots of storage but doesnt need to be that fast)
[21:42] * rendar (~I@host230-177-dynamic.42-79-r.retail.telecomitalia.it) has joined #ceph
[21:43] <lookcrabs> sorry for the barrage of questions but 48 threads/24 cores with 70 drives per box? How are you handling failure? I tried 72 disk SM servers with around 48 threads as well and met with a bit of disaster. Speed was not at all an option for me when I tried.
[21:44] <lookcrabs> and recovery was hellish as the load shot up to thousands pretty quickly without tuning everything down.
[21:45] * olid1119 (~olid1982@aftr-185-17-206-128.dynamic.mnet-online.de) has joined #ceph
[21:45] <lookcrabs> it was only a 1PB test but it didn't sit well with me. We are now using 20 disks per server and only have a 5PB cluster but it seems to be much better. Just curious how you manned the seas of the 70+ disk storage node
[21:46] <mfa298> lookcrabs: mostly by being very careful. We've got most of the recovery settings set very low by default, then push up via injectargs if needed
[21:46] <T1> it must be terrible is a whole node goes down
[21:46] <T1> if even
[21:46] <lookcrabs> indeed
[21:46] <T1> even with 10gbit connectivity it would take forever to recover
[21:47] <lookcrabs> hopefully it is a LACP or 40G backend as 10G wouldn't happen
[21:47] <T1> unless it's not a 1:1 ratio of OSDs and disks
[21:47] <lookcrabs> even then what about the HBA busses for 70+ disks that doesn't sound fun either.
[21:47] <lookcrabs> i assumed with jbod that it is 1/1 but yeah indeed.
[21:47] <T1> heh, yeah
[21:48] <T1> well.. I guess you can get some way with sas multipliers
[21:48] <mfa298> as it's SMR drives we don't push much network traffic most of the time. Reading and testing has suggested it's something like 7iops (write) per drive once the cache is filled
[21:48] * olid1118 (~olid1982@aftr-185-17-206-128.dynamic.mnet-online.de) Quit (Ping timeout: 480 seconds)
[21:49] <mfa298> 45 non SMR drives on those boxes seemed to go quite nicely in a small test (5 nodes so 225 OSDs)
[21:51] <mfa298> the only thing the SMR drives are good for are $/TB and read speeds.
[21:54] * mtb` (~mtb`@157.130.171.46) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[21:54] * dyasny (~dyasny@198.251.57.99) Quit (Ping timeout: 480 seconds)
[21:54] * EinstCrazy (~EinstCraz@117.13.201.130) has joined #ceph
[21:55] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[21:56] * mtb` (~mtb`@157.130.171.46) has joined #ceph
[21:57] * evilrob (~evilrob@2600:3c00::f03c:91ff:fedf:1d3d) has joined #ceph
[21:57] * mtb` (~mtb`@157.130.171.46) Quit ()
[21:58] * mtb` (~mtb`@157.130.171.46) has joined #ceph
[22:00] <lookcrabs> mfa298: are you just using rbd? Do you have a cache tier + EC to get even more or are you using replicas?
[22:01] <mfa298> just rados and replicas, no rbd, EC etc.
[22:01] <lookcrabs> wow raw rados? that's awesome. Just a lowly sysadmin here.
[22:02] <mfa298> been testing various options with SSDs to try and speed things up although nothing conclusuve yet.
[22:02] * lpabon (~quassel@24-151-54-34.dhcp.nwtn.ct.charter.com) has joined #ceph
[22:02] * mtb` (~mtb`@157.130.171.46) Quit ()
[22:03] <mfa298> we've just got lots of fairly small files to store (lots being several billion)
[22:03] * EinstCrazy (~EinstCraz@117.13.201.130) Quit (Ping timeout: 480 seconds)
[22:03] <mfa298> we looked at rgw initially but it doesn't really scale for us.
[22:04] <T1> rbd is a nice abstraction on top of rados
[22:04] <T1> but it is worryingly easy to mess up rbd images with a few stray rados commands.. ;)
[22:05] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) has joined #ceph
[22:06] * boredatwork (~overonthe@199.68.193.54) has joined #ceph
[22:07] * rapedex (~WedTM@6YRAAA9NH.tor-irc.dnsbl.oftc.net) Quit ()
[22:07] * dyasny (~dyasny@198.251.57.99) has joined #ceph
[22:08] * bjozet (~bjozet@82-183-17-144.customers.ownit.se) Quit (Ping timeout: 480 seconds)
[22:12] * ngoswami (~ngoswami@1.39.17.122) Quit (Ping timeout: 480 seconds)
[22:12] * ngoswami (~ngoswami@1.39.97.18) has joined #ceph
[22:13] * dynamicudpate (~overonthe@199.68.193.62) Quit (Ping timeout: 480 seconds)
[22:14] * dynamicudpate (~overonthe@199.68.193.54) has joined #ceph
[22:15] * boredatwork (~overonthe@199.68.193.54) Quit (Ping timeout: 480 seconds)
[22:15] * georgem (~Adium@206.108.127.16) has left #ceph
[22:20] * Wielebny (~Icedove@cl-927.waw-01.pl.sixxs.net) has joined #ceph
[22:21] * johnavp1989 (~jpetrini@8.39.115.8) Quit (Ping timeout: 480 seconds)
[22:23] * bjozet (~bjozet@82-183-17-144.customers.ownit.se) has joined #ceph
[22:27] <rkeene> Hmm, I have a "ceph-mds" server up and running and it is connected to my ceph-mon, but it's not showing up
[22:28] * dnovosel (87f5300c@107.161.19.109) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[22:29] <KaneK> is there anything special to rbd reclaiming the space?
[22:29] <KaneK> ceph health
[22:29] <KaneK> HEALTH_ERR 3 full osd(s)
[22:32] * ngoswami (~ngoswami@1.39.97.18) Quit (Ping timeout: 480 seconds)
[22:34] * dnovosel (87f5300c@107.161.19.53) has joined #ceph
[22:35] <KaneK> also where are osd logs? looks like all logs in /var/log/ceph/osd are empty
[22:36] <KaneK> getting: cluster [ERR] OSD full dropping all updates in mon logs
[22:39] <rkeene> Ah, there were no ceph fs's
[22:40] * reed (~reed@216.38.134.18) Quit (Ping timeout: 480 seconds)
[22:41] * vbellur (~vijay@nat-pool-bos-u.redhat.com) Quit (Ping timeout: 480 seconds)
[22:42] * ngoswami (~ngoswami@1.39.17.122) has joined #ceph
[22:44] * diegows (~diegows@190.190.21.75) has joined #ceph
[22:46] * ngoswami (~ngoswami@1.39.17.122) Quit ()
[22:49] * reed (~reed@216.38.134.18) has joined #ceph
[22:50] * jdillaman (~jdillaman@pool-108-18-97-82.washdc.fios.verizon.net) Quit (Quit: jdillaman)
[22:53] * samx1 (~Adium@12.206.204.58) has joined #ceph
[22:59] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Remote host closed the connection)
[23:00] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[23:00] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[23:05] * phyphor (~LorenXo@89.248.162.153) has joined #ceph
[23:10] <lookcrabs> Does anyone know if ceph S3 supports groups in anyway. IE group ACLs or anything like that?
[23:13] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) Quit (Ping timeout: 480 seconds)
[23:15] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[23:16] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[23:22] * ircolle (~ircolle@nat-pool-bos-u.redhat.com) has left #ceph
[23:24] * nardial (~ls@dslb-084-063-136-171.084.063.pools.vodafone-ip.de) has joined #ceph
[23:28] * KaneK (~kane@cpe-76-89-193-31.socal.res.rr.com) Quit (Quit: KaneK)
[23:31] * johnavp1989 (~jpetrini@pool-100-14-5-21.phlapa.fios.verizon.net) has joined #ceph
[23:33] * thomnico (~thomnico@2a01:e35:8b41:120:84aa:2e97:b789:9f66) has joined #ceph
[23:35] * phyphor (~LorenXo@5P6AABRUY.tor-irc.dnsbl.oftc.net) Quit ()
[23:40] * dnovosel (87f5300c@107.161.19.53) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[23:42] * lpabon (~quassel@24-151-54-34.dhcp.nwtn.ct.charter.com) Quit (Remote host closed the connection)
[23:44] <snakamoto> lookcrabs: I have not seen any 'group' items for bucket ACLs
[23:46] <snakamoto> I can only think of accomplishing something similar by using multiple key/secret pairs and treating the user as a group
[23:52] * wushudoin (~wushudoin@38.140.108.2) Quit (Ping timeout: 480 seconds)
[23:58] * Sami345 (~W|ldCraze@89.248.172.147) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.