#ceph IRC Log

Index

IRC Log for 2015-08-07

Timestamps are in GMT/BST.

[0:03] * fdmanana (~fdmanana@bl13-153-23.dsl.telepac.pt) Quit (Ping timeout: 480 seconds)
[0:08] * badone_ is now known as badone
[0:10] * Nacer (~Nacer@80.12.35.28) Quit (Ping timeout: 480 seconds)
[0:13] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) has joined #ceph
[0:22] * rendar (~I@host28-180-dynamic.11-87-r.retail.telecomitalia.it) Quit ()
[0:23] * segutier (~segutier@sfo-vpn1.shawnlower.net) Quit (Ping timeout: 480 seconds)
[0:39] * segutier (~segutier@sfo-vpn1.shawnlower.net) has joined #ceph
[0:39] <cholcombe> gfidente, sounds like you need to add pgs
[0:40] <cholcombe> gfidente, oh sorry it means you have too few copies running around
[0:41] <cholcombe> gfidente, you might want to do a ceph health --detail to get more info about it
[0:41] <cholcombe> or a dump of the placement groups
[0:44] * arbrandes (~arbrandes@187.34.119.247) has joined #ceph
[0:45] * jhova (~textual@241.50.143.24.cm.sunflower.com) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[0:47] * arbrandes (~arbrandes@187.34.119.247) Quit (Read error: Connection reset by peer)
[0:47] * arbrandes_ (~arbrandes@187.34.119.247) has joined #ceph
[0:47] * arbrandes_ is now known as arbrandes
[0:47] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[0:49] * daviddcc (~dcasier@77.151.197.84) Quit (Remote host closed the connection)
[0:50] * Ian2128 (~pico@0.tor.exit.babylon.network) has joined #ceph
[0:53] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Ping timeout: 480 seconds)
[0:55] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[1:02] * ivotron (~ivotron@eduroam-169-233-197-33.ucsc.edu) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[1:06] * logan (~a@216.144.251.246) has joined #ceph
[1:06] * wicope (~wicope@0001fd8a.user.oftc.net) Quit (Read error: Connection reset by peer)
[1:19] * fred`` (fred@earthli.ng) Quit (Quit: +++ATH0)
[1:20] * Ian2128 (~pico@5NZAAF1ZY.tor-irc.dnsbl.oftc.net) Quit ()
[1:20] * vend3r (~verbalins@chomsky.torservers.net) has joined #ceph
[1:23] * adrian15b (~kvirc@213.Red-79-159-227.dynamicIP.rima-tde.net) Quit (Ping timeout: 480 seconds)
[1:27] * segutier (~segutier@sfo-vpn1.shawnlower.net) Quit (Ping timeout: 480 seconds)
[1:29] * kmARC (~kmARC@91.146.191.230) Quit (Ping timeout: 480 seconds)
[1:30] * kmARC_ (~kmARC@91.146.191.230) Quit (Ping timeout: 480 seconds)
[1:31] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[1:40] * lcurtis (~lcurtis@47.19.105.250) Quit (Remote host closed the connection)
[1:45] * zaitcev (~zaitcev@c-76-113-49-212.hsd1.nm.comcast.net) has joined #ceph
[1:46] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[1:47] * ircolle (~Adium@2601:285:201:2bf9:a92d:b3a3:c72:7fca) Quit (Quit: Leaving.)
[1:49] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:50] * vend3r (~verbalins@7R2AADIY0.tor-irc.dnsbl.oftc.net) Quit ()
[1:50] * N3X15 (~offer@freedom.ip-eend.nl) has joined #ceph
[1:50] * zenpac (~zenpac3@66.55.33.66) has joined #ceph
[1:52] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[1:53] * ivotron (~ivotron@c-67-169-145-20.hsd1.ca.comcast.net) has joined #ceph
[1:56] * oms101 (~oms101@p20030057EA097C00EEF4BBFFFE0F7062.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[2:05] * oms101 (~oms101@p20030057EA094900EEF4BBFFFE0F7062.dip0.t-ipconnect.de) has joined #ceph
[2:07] * thansen (~thansen@17.253.sfcn.org) Quit (Quit: Ex-Chat)
[2:08] * npcomp (~npcomp@c-24-126-240-124.hsd1.ga.comcast.net) Quit (Ping timeout: 480 seconds)
[2:08] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) Quit (Ping timeout: 480 seconds)
[2:12] * xarses (~xarses@12.164.168.117) Quit (Ping timeout: 480 seconds)
[2:14] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) Quit (Quit: doppelgrau)
[2:17] * gfidente (~gfidente@0001ef4b.user.oftc.net) Quit (Ping timeout: 480 seconds)
[2:20] * N3X15 (~offer@5NZAAF11U.tor-irc.dnsbl.oftc.net) Quit ()
[2:20] * rf` (~Cue@aurora.enn.lu) has joined #ceph
[2:30] * zenpac (~zenpac3@66.55.33.66) has joined #ceph
[2:34] * fam_away is now known as fam
[2:35] * _karl_ (~karl@kamr.at) Quit (Remote host closed the connection)
[2:35] * _karl (~karl@kamr.at) has joined #ceph
[2:36] * DV_ (~veillard@2001:41d0:a:f29f::1) Quit (Ping timeout: 480 seconds)
[2:44] * yguang11 (~yguang11@66.228.162.44) Quit ()
[2:46] * Amto_res (~amto_res@ks312256.kimsufi.com) Quit (Ping timeout: 480 seconds)
[2:48] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[2:50] * rf` (~Cue@5NZAAF121.tor-irc.dnsbl.oftc.net) Quit ()
[2:50] * Corti^carte (~loft@politkovskaja.torservers.net) has joined #ceph
[2:56] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[2:57] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[3:03] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[3:11] * snakamoto (~Adium@192.16.26.2) Quit (Quit: Leaving.)
[3:11] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[3:12] * zenpac (~zenpac3@66.55.33.66) has joined #ceph
[3:16] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[3:20] * Corti^carte (~loft@5NZAAF13W.tor-irc.dnsbl.oftc.net) Quit ()
[3:22] * zhaochao (~zhaochao@125.39.8.225) has joined #ceph
[3:23] * angdraug (~angdraug@12.164.168.117) Quit (Quit: Leaving)
[3:23] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[3:23] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[3:41] <hemebond> Why is ceph-users mail archives unavailable?
[3:42] <m0zes> http://news.gmane.org/gmane.comp.file-systems.ceph.user
[3:43] <m0zes> seems to work fine for me.
[3:47] * flisky (~Thunderbi@106.39.60.34) has joined #ceph
[3:50] * Jourei (~uhtr5r@tor.laquadrature.net) has joined #ceph
[3:51] * fsimonce (~simon@host249-48-dynamic.53-79-r.retail.telecomitalia.it) Quit (Remote host closed the connection)
[3:53] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[3:53] * kefu (~kefu@114.92.110.67) has joined #ceph
[3:55] * kefu_ (~kefu@117.186.51.192) has joined #ceph
[3:55] <hemebond> Cheers.
[3:55] <hemebond> My email still hasn't been approved.
[3:58] <m0zes> I have no idea how the approval process happens.
[3:58] <rkeene> how -> if :-D
[3:58] * npcomp (~npcomp@c-24-126-240-124.hsd1.ga.comcast.net) has joined #ceph
[3:58] <hemebond> I was afraid of that.
[3:59] * m0zes hasn't had any issues with the mailing list. ??\_(?????????)_/??
[4:01] * kefu (~kefu@114.92.110.67) Quit (Ping timeout: 480 seconds)
[4:03] * lucas1 (~Thunderbi@218.76.52.64) has joined #ceph
[4:05] * npcomp_ (~npcomp@c-24-126-240-124.hsd1.ga.comcast.net) has joined #ceph
[4:07] * npcomp (~npcomp@c-24-126-240-124.hsd1.ga.comcast.net) Quit (Ping timeout: 480 seconds)
[4:07] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[4:10] * zenpac (~zenpac3@66.55.33.66) has joined #ceph
[4:15] * kefu (~kefu@114.92.110.67) has joined #ceph
[4:15] * zhaochao (~zhaochao@125.39.8.225) Quit (Ping timeout: 480 seconds)
[4:20] * Jourei (~uhtr5r@7R2AADI3N.tor-irc.dnsbl.oftc.net) Quit ()
[4:20] * linjan__ (~linjan@176.195.14.90) Quit (Ping timeout: 480 seconds)
[4:20] * zhaochao (~zhaochao@125.39.8.225) has joined #ceph
[4:22] * kefu_ (~kefu@117.186.51.192) Quit (Ping timeout: 480 seconds)
[4:24] * HoboPickle (~Tenk@192.42.116.16) has joined #ceph
[4:30] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[4:31] * kefu (~kefu@114.92.110.67) has joined #ceph
[4:32] * yuan (~yzhou67@shzdmzpr01-ext.sh.intel.com) has joined #ceph
[4:33] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[4:34] * kefu (~kefu@114.92.110.67) has joined #ceph
[4:38] * lcurtis (~lcurtis@ool-44c08556.dyn.optonline.net) has joined #ceph
[4:48] * flisky (~Thunderbi@106.39.60.34) Quit (Quit: flisky)
[4:49] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[4:50] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[4:51] * kefu (~kefu@114.92.110.67) has joined #ceph
[4:54] * HoboPickle (~Tenk@5NZAAF17P.tor-irc.dnsbl.oftc.net) Quit ()
[4:54] * DJComet (~lobstar@212.47.243.191) has joined #ceph
[4:57] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[4:59] * dlan (~dennis@116.228.88.131) Quit (Remote host closed the connection)
[4:59] * dlan (~dennis@116.228.88.131) has joined #ceph
[5:01] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[5:02] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[5:08] * Vacuum_ (~Vacuum@i59F793C7.versanet.de) has joined #ceph
[5:13] * danieagle (~Daniel@187.75.19.50) has joined #ceph
[5:15] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[5:15] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Remote host closed the connection)
[5:15] * Vacuum__ (~Vacuum@88.130.213.71) Quit (Ping timeout: 480 seconds)
[5:16] * arbrandes (~arbrandes@187.34.119.247) Quit (Quit: Leaving)
[5:19] * ircolle (~Adium@2601:285:201:2bf9:259b:c6c7:ec7f:7f12) has joined #ceph
[5:21] * kefu_ (~kefu@117.186.51.192) has joined #ceph
[5:24] * DJComet (~lobstar@5NZAAF18T.tor-irc.dnsbl.oftc.net) Quit ()
[5:24] * Szernex (~KapiteinK@212.7.194.71) has joined #ceph
[5:25] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[5:28] * kefu (~kefu@114.92.110.67) Quit (Ping timeout: 480 seconds)
[5:30] * zenpac (~zenpac3@66.55.33.66) has joined #ceph
[5:34] * ilken (ilk@2602:63:c2a2:af00:45a6:39a0:cf69:a71c) Quit (Remote host closed the connection)
[5:35] * ilken (ilk@2602:63:c2a2:af00:fda8:cd0c:bf79:773c) has joined #ceph
[5:39] * sankarshan (~sankarsha@122.172.166.118) has joined #ceph
[5:39] * shyu (~Shanzhi@119.254.120.66) has joined #ceph
[5:44] * overclk (~overclk@121.244.87.117) has joined #ceph
[5:45] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[5:48] * kefu_ (~kefu@117.186.51.192) Quit (Max SendQ exceeded)
[5:48] * ircolle (~Adium@2601:285:201:2bf9:259b:c6c7:ec7f:7f12) Quit (Quit: Leaving.)
[5:49] * kefu (~kefu@li750-169.members.linode.com) has joined #ceph
[5:54] * Szernex (~KapiteinK@5NZAAF192.tor-irc.dnsbl.oftc.net) Quit ()
[5:54] * MonkeyJamboree (~starcoder@exit.tor.uwaterloo.ca) has joined #ceph
[5:59] * LeaChim (~LeaChim@host86-132-233-87.range86-132.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[6:11] * dyasny (~dyasny@104.158.25.230) Quit (Ping timeout: 480 seconds)
[6:24] * MonkeyJamboree (~starcoder@5NZAAF2A1.tor-irc.dnsbl.oftc.net) Quit ()
[6:24] * PuyoDead (~EdGruberm@tor-exit-node--proxy.scalaire.com) has joined #ceph
[6:28] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) has joined #ceph
[6:35] * hemebond (~james@121.98.129.142) Quit (Ping timeout: 480 seconds)
[6:35] * hemebond (~james@121-98-133-215.bng1.nct.orcon.net.nz) has joined #ceph
[6:38] * segutier (~segutier@65.126.99.11) has joined #ceph
[6:49] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[6:50] * reed (~reed@75-101-54-131.dsl.static.fusionbroadband.com) Quit (Ping timeout: 480 seconds)
[6:50] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[6:51] * ivotron (~ivotron@c-67-169-145-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[6:54] * PuyoDead (~EdGruberm@9S0AAC70D.tor-irc.dnsbl.oftc.net) Quit ()
[6:54] * Peaced (~Silentspy@hessel2.torservers.net) has joined #ceph
[6:58] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[6:58] * reed (~reed@75-101-54-131.dsl.static.fusionbroadband.com) has joined #ceph
[7:02] * kefu (~kefu@li750-169.members.linode.com) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[7:04] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) Quit (Ping timeout: 480 seconds)
[7:09] * zenpac (~zenpac3@66.55.33.66) has joined #ceph
[7:11] * haomaiwang (~haomaiwan@60-250-10-249.HINET-IP.hinet.net) Quit (Quit: Leaving...)
[7:11] * haomaiwang (~haomaiwan@60-250-10-249.HINET-IP.hinet.net) has joined #ceph
[7:18] * reed (~reed@75-101-54-131.dsl.static.fusionbroadband.com) Quit (Ping timeout: 480 seconds)
[7:24] * Peaced (~Silentspy@5NZAAF2C2.tor-irc.dnsbl.oftc.net) Quit ()
[7:24] * Corti^carte (~Moriarty@5.9.158.75) has joined #ceph
[7:29] * kefu (~kefu@117.186.51.192) has joined #ceph
[7:36] * snakamoto (~Adium@cpe-76-91-202-90.socal.res.rr.com) has joined #ceph
[7:40] * snakamoto (~Adium@cpe-76-91-202-90.socal.res.rr.com) Quit ()
[7:45] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[7:45] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[7:46] * zaitcev (~zaitcev@c-76-113-49-212.hsd1.nm.comcast.net) Quit (Quit: Bye)
[7:49] * rdas (~rdas@121.244.87.116) has joined #ceph
[7:53] * segutier (~segutier@65.126.99.11) Quit (Quit: segutier)
[7:54] * Corti^carte (~Moriarty@9S0AAC715.tor-irc.dnsbl.oftc.net) Quit ()
[7:54] * Joppe4899 (~Sliker@manning1.torservers.net) has joined #ceph
[8:06] * lcurtis (~lcurtis@ool-44c08556.dyn.optonline.net) Quit (Ping timeout: 480 seconds)
[8:11] * zenpac (~zenpac3@66.55.33.66) Quit (Read error: Connection reset by peer)
[8:16] * shang (~ShangWu@175.41.48.77) has joined #ceph
[8:20] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[8:24] * Joppe4899 (~Sliker@9S0AAC72X.tor-irc.dnsbl.oftc.net) Quit ()
[8:24] * dotblank (~Diablothe@7R2AADJA5.tor-irc.dnsbl.oftc.net) has joined #ceph
[8:24] * segutier (~segutier@65.126.99.11) has joined #ceph
[8:28] * dopesong (~dopesong@lb1.mailer.data.lt) has joined #ceph
[8:37] * saltsa_ (~joonas@dsl-hkibrasgw1-58c018-65.dhcp.inet.fi) Quit (Ping timeout: 480 seconds)
[8:39] * kawa2014 (~kawa@89.184.114.246) has joined #ceph
[8:41] * segutier (~segutier@65.126.99.11) Quit (Quit: segutier)
[8:44] * overclk (~overclk@121.244.87.117) Quit (Quit: Leaving)
[8:50] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[8:51] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[8:51] * dgurtner (~dgurtner@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[8:52] * derjohn_mob (~aj@x590e0906.dyn.telefonica.de) has joined #ceph
[8:54] * dotblank (~Diablothe@7R2AADJA5.tor-irc.dnsbl.oftc.net) Quit ()
[8:54] * Mraedis (~lobstar@lumumba.torservers.net) has joined #ceph
[8:56] * saltsa (~joonas@dsl-hkibrasgw1-58c018-65.dhcp.inet.fi) has joined #ceph
[8:57] * shang (~ShangWu@175.41.48.77) Quit (Ping timeout: 480 seconds)
[8:59] * Be-El (~quassel@fb08-bcf-pc01.computational.bio.uni-giessen.de) has joined #ceph
[8:59] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[8:59] <Be-El> hi
[9:04] * rendar (~I@host238-62-dynamic.247-95-r.retail.telecomitalia.it) has joined #ceph
[9:05] * Guest1930 is now known as winston-d
[9:06] * sleinen (~Adium@172.113.77.83.dynamic.wline.res.cust.swisscom.ch) has joined #ceph
[9:06] * shang (~ShangWu@175.41.48.77) has joined #ceph
[9:06] * adrian15b (~kvirc@189.Red-83-32-56.dynamicIP.rima-tde.net) has joined #ceph
[9:07] * sleinen1 (~Adium@2001:620:0:82::101) has joined #ceph
[9:08] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[9:10] * b0e (~aledermue@213.95.25.82) has joined #ceph
[9:10] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) has joined #ceph
[9:14] * sleinen (~Adium@172.113.77.83.dynamic.wline.res.cust.swisscom.ch) Quit (Ping timeout: 480 seconds)
[9:19] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[9:19] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[9:20] * neurodrone (~neurodron@108.60.145.130) Quit (Ping timeout: 480 seconds)
[9:24] * Mraedis (~lobstar@5NZAAF2H6.tor-irc.dnsbl.oftc.net) Quit ()
[9:24] * xolotl (~SEBI@atlantic480.us.unmetered.com) has joined #ceph
[9:29] * shang (~ShangWu@175.41.48.77) Quit (Ping timeout: 480 seconds)
[9:33] * adrian15b (~kvirc@189.Red-83-32-56.dynamicIP.rima-tde.net) Quit (Ping timeout: 480 seconds)
[9:37] * linjan__ (~linjan@195.91.236.115) has joined #ceph
[9:41] * shang (~ShangWu@175.41.48.77) has joined #ceph
[9:43] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[9:43] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Remote host closed the connection)
[9:49] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[9:50] * branto (~branto@213.175.37.10) has joined #ceph
[9:54] * macjack (~macjack@122.146.93.152) has left #ceph
[9:54] * xolotl (~SEBI@9S0AAC75R.tor-irc.dnsbl.oftc.net) Quit ()
[9:54] * Nacer (~Nacer@80.12.51.186) has joined #ceph
[10:02] * fdmanana (~fdmanana@bl13-145-124.dsl.telepac.pt) has joined #ceph
[10:06] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[10:06] * analbeard (~shw@support.memset.com) has joined #ceph
[10:07] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[10:07] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[10:09] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[10:09] * campee (~campee@c-50-148-149-27.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[10:13] * chungting (~chungting@122.146.93.152) Quit (Quit: Leaving)
[10:14] * a1-away (~jelle@62.27.85.48) Quit (Ping timeout: 480 seconds)
[10:15] * dis (~dis@109.110.66.238) Quit (Ping timeout: 480 seconds)
[10:21] * dgurtner (~dgurtner@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[10:21] * derjohn_mob (~aj@x590e0906.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[10:22] * shang (~ShangWu@175.41.48.77) Quit (Remote host closed the connection)
[10:24] * jordanP (~jordan@213.215.2.194) has joined #ceph
[10:24] * _s1gma (~legion@5.9.158.75) has joined #ceph
[10:25] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[10:25] * a1-away (~jelle@62.27.85.48) has joined #ceph
[10:32] * wicope (~wicope@0001fd8a.user.oftc.net) has joined #ceph
[10:39] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) has joined #ceph
[10:41] * tjt263 (~RAPSCA77I@106-68-45-204.dyn.iinet.net.au) has joined #ceph
[10:48] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) Quit (Quit: doppelgrau)
[10:49] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Quit: Hiring PHP developers does not contribute to the quota of employees with disabilities.)
[10:50] * KindOne (kindone@0001a7db.user.oftc.net) has joined #ceph
[10:52] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[10:54] * _s1gma (~legion@7R2AADJEU.tor-irc.dnsbl.oftc.net) Quit ()
[10:54] * Diablodoct0r (~uhtr5r@178-175-128-50.ip.as43289.net) has joined #ceph
[10:58] * derjohn_mob (~aj@2001:6f8:1337:0:e548:f75f:2fa7:8f77) has joined #ceph
[11:00] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[11:00] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[11:01] * vikhyat (~vumrao@121.244.87.116) has joined #ceph
[11:04] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) has joined #ceph
[11:07] * imcsk8 (~ichavero@189.231.89.185) Quit (Ping timeout: 480 seconds)
[11:15] * dgurtner (~dgurtner@178.197.231.115) has joined #ceph
[11:18] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[11:19] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[11:19] * adrian15b (~kvirc@btactic.ddns.jazztel.es) has joined #ceph
[11:23] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[11:24] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) Quit (Read error: No route to host)
[11:24] * Diablodoct0r (~uhtr5r@5NZAAF2RL.tor-irc.dnsbl.oftc.net) Quit ()
[11:24] * vend3r (~djidis__@5NZAAF2TX.tor-irc.dnsbl.oftc.net) has joined #ceph
[11:25] * brutusca_ (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) has joined #ceph
[11:32] * ninkotech (~duplo@static-84-242-87-186.net.upcbroadband.cz) has joined #ceph
[11:39] * bara (~bara@ip4-83-240-10-82.cust.nbox.cz) has joined #ceph
[11:45] * nardial (~ls@dslb-178-006-188-098.178.006.pools.vodafone-ip.de) has joined #ceph
[11:48] * campee (~campee@c-50-148-149-27.hsd1.ca.comcast.net) has joined #ceph
[11:54] * vend3r (~djidis__@5NZAAF2TX.tor-irc.dnsbl.oftc.net) Quit ()
[11:54] * fabioFVZ (~fabiofvz@213.187.10.8) has joined #ceph
[11:54] * cheese^ (~lmg@edwardsnowden0.torservers.net) has joined #ceph
[11:54] * fabioFVZ (~fabiofvz@213.187.10.8) Quit ()
[11:55] * nardial (~ls@dslb-178-006-188-098.178.006.pools.vodafone-ip.de) Quit (Read error: No route to host)
[11:56] * __NiC (~kristian@aeryn.ronningen.no) Quit (Quit: leaving)
[11:56] * _NiC (~kristian@aeryn.ronningen.no) has joined #ceph
[12:00] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[12:00] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[12:04] * Nacer_ (~Nacer@80.12.35.3) has joined #ceph
[12:05] * nardial (~ls@dslb-178-006-188-098.178.006.pools.vodafone-ip.de) has joined #ceph
[12:05] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[12:06] * Nacer (~Nacer@80.12.51.186) Quit (Ping timeout: 480 seconds)
[12:08] * brutusca_ (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) Quit (Read error: Connection reset by peer)
[12:09] * kmARC (~kmARC@91.146.191.230) has joined #ceph
[12:10] * kmARC_ (~kmARC@91.146.191.230) has joined #ceph
[12:12] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) has joined #ceph
[12:13] <vikhyat> ping ceph : Suppose we have 100 objects 1MB each vs 1 million small objects, in this case is it better to have say 100 PGs or more like 600 PGs since the objects are smaller?
[12:14] <kefu> vikhyat: have you checked http://ceph.com/pgcalc/ ?
[12:14] <kefu> just a quick q.
[12:14] <vikhyat> kefu: yup
[12:15] <vikhyat> kefu: actually it does not have object size so thought lets check
[12:15] * Nacer_ (~Nacer@80.12.35.3) Quit (Ping timeout: 480 seconds)
[12:18] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[12:18] * zhaochao (~zhaochao@125.39.8.225) Quit (Quit: ChatZilla 0.9.91.1 [Iceweasel 38.1.0/20150711212448])
[12:24] * cheese^ (~lmg@5NZAAF2VV.tor-irc.dnsbl.oftc.net) Quit ()
[12:24] * toast (~Zombiekil@192.42.116.16) has joined #ceph
[12:26] * nardial (~ls@dslb-178-006-188-098.178.006.pools.vodafone-ip.de) Quit (Quit: Leaving)
[12:26] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[12:26] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[12:27] * shyu (~Shanzhi@119.254.120.66) Quit (Remote host closed the connection)
[12:31] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[12:34] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[12:40] * lucas1 (~Thunderbi@218.76.52.64) Quit (Quit: lucas1)
[12:41] * lucas1 (~Thunderbi@218.76.52.64) has joined #ceph
[12:43] * derjohn_mob (~aj@2001:6f8:1337:0:e548:f75f:2fa7:8f77) Quit (Ping timeout: 480 seconds)
[12:48] * rotbeard (~redbeard@cm-171-100-223-199.revip10.asianet.co.th) has joined #ceph
[12:52] * alexbligh1 (~alexbligh@89-16-176-215.no-reverse-dns-set.bytemark.co.uk) Quit (Ping timeout: 480 seconds)
[12:52] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[12:54] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[12:54] * gregmark (~Adium@68.87.42.115) Quit (Read error: Connection reset by peer)
[12:54] * toast (~Zombiekil@7R2AADJIF.tor-irc.dnsbl.oftc.net) Quit ()
[12:55] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[12:56] * gregmark (~Adium@68.87.42.115) has joined #ceph
[12:56] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[12:59] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) Quit (Remote host closed the connection)
[12:59] * Tene (~tene@173.13.139.236) Quit (Remote host closed the connection)
[12:59] * Tene (~tene@173.13.139.236) has joined #ceph
[13:00] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[13:01] * alexbligh1 (~alexbligh@89-16-176-215.no-reverse-dns-set.bytemark.co.uk) has joined #ceph
[13:01] * eternaleye (~eternaley@50.245.141.73) Quit (Ping timeout: 480 seconds)
[13:01] * championofcyrodi (~championo@50-205-35-98-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[13:02] * kmARC_ (~kmARC@91.146.191.230) Quit (Ping timeout: 480 seconds)
[13:02] * kmARC (~kmARC@91.146.191.230) Quit (Ping timeout: 480 seconds)
[13:03] * championofcyrodi (~championo@50-205-35-98-static.hfc.comcastbusiness.net) has joined #ceph
[13:04] * eternaleye (~eternaley@50.245.141.73) has joined #ceph
[13:04] * daviddcc (~dcasier@LAubervilliers-656-1-16-164.w217-128.abo.wanadoo.fr) has joined #ceph
[13:05] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has joined #ceph
[13:07] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[13:07] * derjohn_mob (~aj@2001:6f8:1337:0:7d9d:66bd:2315:d5c6) has joined #ceph
[13:11] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[13:13] * lucas1 (~Thunderbi@218.76.52.64) Quit (Quit: lucas1)
[13:16] * yanzheng (~zhyan@182.139.23.163) Quit ()
[13:19] * kefu is now known as kefu|afk
[13:21] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[13:24] * shaunm (~shaunm@80.252.214.146) has joined #ceph
[13:24] * Corti^carte (~hoopy@104.238.169.63) has joined #ceph
[13:28] * nadav (~oftc-webi@216.223.13.181) Quit (Ping timeout: 480 seconds)
[13:30] * kefu|afk (~kefu@117.186.51.192) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[13:31] * alfredodeza (~alfredode@198.206.133.89) has joined #ceph
[13:35] * arbrandes (~arbrandes@187.34.119.247) has joined #ceph
[13:36] <arbrandes> Hey guys, how's it going.
[13:36] <arbrandes> Little advice needed:
[13:36] <arbrandes> I need a way, if at all possible, to zap and recreate an OSD without "outting" it. Is this at all possible?
[13:37] <arbrandes> I really do need to zap it, btw. I've hit on what I believe is a btfs-related data corruption issue that only goes away with a complete rebuild of the OSD.
[13:38] <arbrandes> But the issue is so prickly that if data gets shuffled around to other 0.94.2 OSDs (I'm in the middle of an upgrade from 0.93), THEY'll also croak.
[13:38] <arbrandes> This is being tracked on #12194, btw.
[13:39] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[13:39] <Be-El> arbrandes: you can use noout,nodown,nobackfill,norecovery flags to prevent ceph from creating new data copies after the osd is offline
[13:40] <Be-El> arbrandes: but i
[13:40] <Be-El> arbrandes: but i'm just another user, i haven't tested it before
[13:40] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[13:41] <arbrandes> Be-El, thanks, I'll certainly try those!
[13:41] <Be-El> arbrandes: if you are using btrfs, you might want to disable snapshots
[13:41] <Be-El> arbrandes: they have been unreliable on our osds in the past
[13:42] <arbrandes> Be-El, actually, I'm no longer using btrfs. I successfully (it seemed) converted all the OSDs to xfs, while running 0.93.
[13:42] <arbrandes> Be-El, I did it the Ugly Way, though. rsync -aX data out, recreate the filesystem manually with xfs, rsync-aX data back in.
[13:42] <arbrandes> Be-El, I believe this has something to do with the bug I'm hitting now.
[13:44] <T1w> uff.. seems like I'm not wrong in choosing xfs for our new setup
[13:44] <T1w> on stock rhel7
[13:44] <T1w> over time perhaps on stock centos7
[13:44] <Be-El> T1w: there are pro and cons for xfs and btrfs
[13:45] <Be-El> T1w: i like the checksum feature of btrfs that tells you when some data has been rotten
[13:45] <T1w> yeah, but the small amount of metadata that can be stored in btrfs doesn't outweigh the risks
[13:45] <arbrandes> The problem with btrfs is this: you have to be really, REALLY, careful in micromanaging the filesystems, which is, as far as I'm concerned, completely counterproductive.
[13:45] <Be-El> metadata? you mean the xattr stuff?
[13:46] <T1w> oh yeah - xattr
[13:46] <Be-El> afaik btrfs is not limited on that (in contrast to xfs an ext4)
[13:46] <Be-El> +d
[13:46] <T1w> and since btrfs is still "tech preview" from redhats side I'm not having it come near my production env
[13:46] <Be-El> but xfs is probably the safest choice today
[13:47] <T1w> I seem to remember that btrfs could store more than xfs without having to overflow it to files
[13:47] <T1w> but I could be wrong.. :)
[13:47] <arbrandes> Be-El, out of curiosity: do you run rebalance operations manually/periodically on your btrfs filesystems?
[13:48] <arbrandes> Be-El, and how full do you let them get, at most (and I mean the total "btrfs fi show" gives, not df)?
[13:48] <Tetard> ""Some people are already using it in production; we're terrified of this. It's really not ready yet."
[13:48] <T1w> Tetard: wheres that from?
[13:48] <Tetard> ah, that's a quote about cephfs, not btrfs
[13:48] <Tetard> http://www.zdnet.com/article/linux-storage-futures/
[13:49] <T1w> ah
[13:49] <Tetard> my bad :)
[13:49] <T1w> almost the same thing IMO.. ;)
[13:49] <Tetard> haha
[13:49] <T1w> oh well..
[13:49] <Tetard> but btrfs isn't production ready
[13:49] <Be-El> arbrandes: usually manually. it is also only needed for older kernels (e.g. ubuntu trusty's default kernel)
[13:49] <T1w> some time next week I'll start performancetesting our expected ceph-node config
[13:49] <Tetard> (unless it's a very narrow use case where you're sure not to use all the features)
[13:50] <T1w> I'll see what happens
[13:51] * Hemanth (~Hemanth@121.244.87.117) has joined #ceph
[13:52] <arbrandes> Be-El, yeah, we ran into a situation where a pre-3.18 kernel filled up all our btrfs OSDs, to the extent that a rebalance operation was taking forever and doing nothing.
[13:54] * Corti^carte (~hoopy@104.238.169.63) Quit ()
[13:54] * Kakeru (~blank@chomsky.torservers.net) has joined #ceph
[14:00] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[14:00] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[14:00] * lcurtis (~lcurtis@ool-44c08556.dyn.optonline.net) has joined #ceph
[14:00] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[14:01] * visbits (~textual@8.29.138.28) Quit (Quit: Textual IRC Client: www.textualapp.com)
[14:05] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[14:08] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) Quit (Remote host closed the connection)
[14:09] * lcurtis (~lcurtis@ool-44c08556.dyn.optonline.net) Quit (Ping timeout: 480 seconds)
[14:10] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) has joined #ceph
[14:11] * badone (~brad@CPE-121-215-241-179.static.qld.bigpond.net.au) Quit (Ping timeout: 480 seconds)
[14:13] * kefu (~kefu@117.186.51.192) has joined #ceph
[14:16] <arbrandes> A question about "nodown": will client traffic be directed at a down OSD, and thus hang or fail indefinitely?
[14:18] * shohn (~shohn@p57A15B83.dip0.t-ipconnect.de) has joined #ceph
[14:19] <arbrandes> Apparently, yes. :) Thanks leseb_. http://www.sebastien-han.fr/blog/2013/04/17/some-ceph-experiments/
[14:19] <leseb_> arbrandes: ;-)
[14:24] * Kakeru (~blank@9S0AAC8ES.tor-irc.dnsbl.oftc.net) Quit ()
[14:24] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) has joined #ceph
[14:24] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[14:25] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[14:33] * vikhyat (~vumrao@121.244.87.116) Quit (Quit: Leaving)
[14:35] * overclk (~overclk@59.93.224.138) has joined #ceph
[14:37] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[14:40] * shaunm (~shaunm@80.252.214.146) Quit (Ping timeout: 480 seconds)
[14:51] * dis (~dis@109.110.66.71) has joined #ceph
[14:51] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[14:52] * Hemanth (~Hemanth@121.244.87.117) Quit (Ping timeout: 480 seconds)
[14:53] * analbeard (~shw@support.memset.com) Quit (Remote host closed the connection)
[14:53] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[14:54] * overclk_ (~overclk@59.93.224.138) has joined #ceph
[14:54] * analbeard (~shw@support.memset.com) has joined #ceph
[14:54] * BillyBobJohn (~Malcovent@9S0AAC8G1.tor-irc.dnsbl.oftc.net) has joined #ceph
[14:58] * overclk_ (~overclk@59.93.224.138) Quit (Remote host closed the connection)
[15:00] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) has joined #ceph
[15:01] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[15:01] * overclk (~overclk@59.93.224.138) Quit (Ping timeout: 480 seconds)
[15:10] * dneary (~dneary@nat-pool-bos-u.redhat.com) has joined #ceph
[15:18] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) has joined #ceph
[15:23] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) has joined #ceph
[15:23] * brad_mssw (~brad@66.129.88.50) has joined #ceph
[15:24] * BillyBobJohn (~Malcovent@9S0AAC8G1.tor-irc.dnsbl.oftc.net) Quit ()
[15:24] * superdug (~anadrom@marylou.nos-oignons.net) has joined #ceph
[15:27] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[15:32] * yanzheng (~zhyan@182.139.23.163) Quit (Quit: This computer has gone to sleep)
[15:36] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) Quit (Read error: No route to host)
[15:37] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) has joined #ceph
[15:38] * yanzheng (~zhyan@182.139.23.163) has joined #ceph
[15:38] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[15:38] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[15:42] * harold (~hamiller@71-94-227-123.dhcp.mdfd.or.charter.com) has joined #ceph
[15:43] * erice (~erice@c-76-120-53-165.hsd1.co.comcast.net) has joined #ceph
[15:46] * jrankin (~jrankin@d53-64-170-236.nap.wideopenwest.com) has joined #ceph
[15:46] * dyasny (~dyasny@173.231.115.59) has joined #ceph
[15:49] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[15:53] * kefu is now known as kefu|afk
[15:54] * kefu|afk is now known as kefu
[15:54] * superdug (~anadrom@7R2AADJQX.tor-irc.dnsbl.oftc.net) Quit ()
[15:54] * ricin (~osuka_@freedom.ip-eend.nl) has joined #ceph
[15:59] * overclk (~overclk@117.208.188.90) has joined #ceph
[16:07] * overclk (~overclk@117.208.188.90) Quit (Ping timeout: 480 seconds)
[16:15] * fam is now known as fam_away
[16:21] * branto (~branto@213.175.37.10) Quit (Quit: Leaving.)
[16:23] * harold (~hamiller@71-94-227-123.dhcp.mdfd.or.charter.com) Quit (Quit: Leaving)
[16:23] * lcurtis (~lcurtis@47.19.105.250) has joined #ceph
[16:23] * linjan__ (~linjan@195.91.236.115) Quit (Ping timeout: 480 seconds)
[16:24] * ricin (~osuka_@7R2AADJSE.tor-irc.dnsbl.oftc.net) Quit ()
[16:24] * DoDzy (~LorenXo@192.42.116.16) has joined #ceph
[16:34] * Nacer (~Nacer@80.12.51.201) has joined #ceph
[16:37] * derjohn_mob (~aj@2001:6f8:1337:0:7d9d:66bd:2315:d5c6) Quit (Ping timeout: 480 seconds)
[16:38] * Azylog (~pandemik@2a01:6380:fff9:2f:1d7a:ae9:2ed4:e709) Quit (Quit: Quitte)
[16:42] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) Quit (Read error: No route to host)
[16:43] * Nacer (~Nacer@80.12.51.201) Quit (Ping timeout: 480 seconds)
[16:43] * Nacer (~Nacer@80.12.35.152) has joined #ceph
[16:45] * Nacer_ (~Nacer@37.163.21.213) has joined #ceph
[16:47] * b0e (~aledermue@213.95.25.82) Quit (Quit: Leaving.)
[16:48] * amote (~amote@121.244.87.116) Quit (Quit: Leaving)
[16:51] * vbellur (~vijay@122.172.220.185) has joined #ceph
[16:52] * reed (~reed@75-101-54-131.dsl.static.fusionbroadband.com) has joined #ceph
[16:52] * Nacer (~Nacer@80.12.35.152) Quit (Ping timeout: 480 seconds)
[16:54] * DoDzy (~LorenXo@9S0AAC8LG.tor-irc.dnsbl.oftc.net) Quit ()
[16:55] <ska> Is there a Ceph-OpenStack-Calamari version compatibility chart of any kind?
[16:58] * LeaChim (~LeaChim@host86-132-233-87.range86-132.btcentralplus.com) has joined #ceph
[16:59] <ska> Also, is the upgrade freqency and path for Ceph going to be higher than for something like OpenStack?
[17:00] * georgem (~Adium@206.108.127.16) has joined #ceph
[17:06] * vbellur (~vijay@122.172.220.185) Quit (Ping timeout: 480 seconds)
[17:14] * yguang11 (~yguang11@2001:4998:effd:600:55cb:bfda:a92b:bc9) has joined #ceph
[17:17] * branto (~branto@ip-213-220-232-132.net.upcbroadband.cz) has joined #ceph
[17:18] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) has joined #ceph
[17:18] * vbellur (~vijay@122.172.46.181) has joined #ceph
[17:25] * hassifa (~rhonabwy@hessel2.torservers.net) has joined #ceph
[17:32] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[17:33] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[17:36] * kefu is now known as kefu|afk
[17:36] * sankarshan (~sankarsha@122.172.166.118) Quit (Quit: Are you sure you want to quit this channel (Cancel/Ok) ?)
[17:36] * jclm1 (~jclm@ip68-108-160-186.lv.lv.cox.net) has joined #ceph
[17:37] * elder (~elder@c-24-245-18-91.hsd1.mn.comcast.net) Quit (Quit: Leaving)
[17:38] * kefu|afk is now known as kefu
[17:39] * derjohn_mob (~aj@b2b-94-79-172-98.unitymedia.biz) has joined #ceph
[17:39] * jclm (~jclm@ip68-108-160-186.lv.lv.cox.net) Quit (Ping timeout: 480 seconds)
[17:40] * elder (~elder@c-24-245-18-91.hsd1.mn.comcast.net) has joined #ceph
[17:40] * ChanServ sets mode +o elder
[17:41] * logan (~a@216.144.251.246) Quit (Ping timeout: 480 seconds)
[17:41] * logan (~a@216.144.251.246) has joined #ceph
[17:42] * georgem (~Adium@206.108.127.16) has joined #ceph
[17:44] * georgem (~Adium@206.108.127.16) has left #ceph
[17:44] * Nacer_ (~Nacer@37.163.21.213) Quit (Ping timeout: 480 seconds)
[17:44] * jclm (~jclm@ip68-108-160-186.lv.lv.cox.net) has joined #ceph
[17:48] * xabner (~xabner@2607:f388:1090:0:bd7f:79ca:50eb:f08) has joined #ceph
[17:48] * jclm1 (~jclm@ip68-108-160-186.lv.lv.cox.net) Quit (Ping timeout: 480 seconds)
[17:49] * derjohn_mob (~aj@b2b-94-79-172-98.unitymedia.biz) Quit (Ping timeout: 480 seconds)
[17:52] * CheKoLyN (~saguilar@bender.parc.xerox.com) has joined #ceph
[17:54] * segutier (~segutier@sfo-vpn1.shawnlower.net) has joined #ceph
[17:54] * hassifa (~rhonabwy@9S0AAC8OZ.tor-irc.dnsbl.oftc.net) Quit ()
[17:54] * rapedex (~WedTM@ns316491.ip-37-187-129.eu) has joined #ceph
[17:55] * TheSov (~TheSov@cip-248.trustwave.com) has joined #ceph
[17:59] * kefu (~kefu@117.186.51.192) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[17:59] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[17:59] * kefu (~kefu@114.92.110.67) has joined #ceph
[17:59] * kefu is now known as kefu|afk
[18:00] * shaunm (~shaunm@80.252.214.146) has joined #ceph
[18:00] * kefu|afk is now known as kefu
[18:05] * dopesong (~dopesong@lb1.mailer.data.lt) Quit (Ping timeout: 480 seconds)
[18:06] * georgem (~Adium@206.108.127.16) has joined #ceph
[18:08] * jordanP (~jordan@213.215.2.194) Quit (Quit: Leaving)
[18:10] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[18:10] * kefu (~kefu@114.92.110.67) has joined #ceph
[18:11] * daviddcc (~dcasier@LAubervilliers-656-1-16-164.w217-128.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[18:14] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[18:15] * overclk (~overclk@59.93.64.65) has joined #ceph
[18:15] * overclk (~overclk@59.93.64.65) Quit (autokilled: This host may be infected. Mail support@oftc.net with questions. BOPM (2015-08-07 16:15:38))
[18:24] * rapedex (~WedTM@9S0AAC8QC.tor-irc.dnsbl.oftc.net) Quit ()
[18:29] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[18:30] * rdas (~rdas@121.244.87.116) has joined #ceph
[18:30] * rdas (~rdas@121.244.87.116) Quit ()
[18:31] * moore (~moore@69.162.16.14) has joined #ceph
[18:31] * moore (~moore@69.162.16.14) Quit (Remote host closed the connection)
[18:32] * moore (~moore@64.202.160.233) has joined #ceph
[18:32] * jrankin (~jrankin@d53-64-170-236.nap.wideopenwest.com) Quit (Quit: Leaving)
[18:33] * elder_ (~elder@apis-host139.dsl.visi.com) has joined #ceph
[18:34] * segutier (~segutier@sfo-vpn1.shawnlower.net) Quit (Ping timeout: 480 seconds)
[18:35] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[18:36] * emre (~edemirors@97.65.225.1) has joined #ceph
[18:36] * kefu (~kefu@114.92.110.67) has joined #ceph
[18:36] <emre> sup folks
[18:36] <emre> anyone tried running ceph mesos framework?
[18:36] <emre> https://github.com/Intel-bigdata/ceph-mesos
[18:36] * shohn (~shohn@p57A15B83.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[18:37] * magicrob1tmonkey is now known as magicrobotmonkey
[18:40] * kawa2014 (~kawa@89.184.114.246) Quit (Quit: Leaving)
[18:43] * branto (~branto@ip-213-220-232-132.net.upcbroadband.cz) Quit (Quit: Leaving.)
[18:43] * bara (~bara@ip4-83-240-10-82.cust.nbox.cz) Quit (Quit: Bye guys! (??????????????????? ?????????)
[18:48] * rotbeard (~redbeard@cm-171-100-223-199.revip10.asianet.co.th) Quit (Ping timeout: 480 seconds)
[18:49] * Nacer (~Nacer@37.163.21.213) has joined #ceph
[18:50] * Be-El (~quassel@fb08-bcf-pc01.computational.bio.uni-giessen.de) Quit (Remote host closed the connection)
[18:51] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[18:53] * shaunm (~shaunm@80.252.214.146) Quit (Ping timeout: 480 seconds)
[18:54] * cedric_ (~cedric@control.slider.be) has joined #ceph
[18:54] * evilrob00 (~evilrob00@cpe-72-179-3-209.austin.res.rr.com) Quit (Read error: Connection reset by peer)
[18:55] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[18:55] * evilrob00 (~evilrob00@cpe-72-179-3-209.austin.res.rr.com) has joined #ceph
[18:57] * kefu (~kefu@114.92.110.67) has joined #ceph
[18:57] * dgbaley27 (~matt@c-67-176-93-83.hsd1.co.comcast.net) has joined #ceph
[18:58] * ircolle (~Adium@2601:285:201:2bf9:259b:c6c7:ec7f:7f12) has joined #ceph
[18:59] <cholcombe> anyone have any dell performance numbers for ceph?
[19:02] <monsted> any dell product in particular?
[19:03] <cholcombe> how about a r730xd with 3.5" drives
[19:03] <cholcombe> monsted, what are you running?
[19:03] <monsted> a hp microserver N40L :)
[19:04] <cholcombe> hmm i'm not familiar with those. let me look
[19:04] <cholcombe> monsted, is it working well for you?
[19:04] <monsted> it's ancient and slow
[19:04] * dgbaley27 (~matt@c-67-176-93-83.hsd1.co.comcast.net) Quit (Remote host closed the connection)
[19:04] <cholcombe> lol
[19:04] * evilrob00 (~evilrob00@cpe-72-179-3-209.austin.res.rr.com) Quit (Ping timeout: 480 seconds)
[19:05] <monsted> but for the purposes of ceph, the badge on the front makes so little difference that you'd be better off comparing the guts of the machine - CPU, RAM, controller and disk is about all that matters
[19:07] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) Quit (Quit: Segmentation fault (core dumped))
[19:07] <monsted> looks like that dell will take 12x 3.5" and 2x 2.5" - should do very well as OSD with two 6-core CPUs
[19:07] * evilrob00 (~evilrob00@cpe-72-179-3-209.austin.res.rr.com) has joined #ceph
[19:09] * xarses (~xarses@12.164.168.117) has joined #ceph
[19:10] * segutier (~segutier@sfo-vpn1.shawnlower.net) has joined #ceph
[19:10] * moore (~moore@64.202.160.233) Quit (Remote host closed the connection)
[19:13] * snakamoto (~Adium@192.16.26.2) has joined #ceph
[19:13] * TMM (~hp@178-84-46-106.dynamic.upc.nl) has joined #ceph
[19:14] * mattrich (~Adium@38.108.161.133) has joined #ceph
[19:17] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) has joined #ceph
[19:20] * brutuscat (~brutuscat@234.Red-79-151-98.dynamicIP.rima-tde.net) Quit (Remote host closed the connection)
[19:24] * Spikey (~Deiz@tor1e1.privacyfoundation.ch) has joined #ceph
[19:24] <burley> cholcombe: We run R720xd's with single E5-2620, 64GB RAM
[19:25] <cholcombe> burley, how are they performing for you under heavy write workloads?
[19:29] * dgurtner (~dgurtner@178.197.231.115) Quit (Ping timeout: 480 seconds)
[19:29] <burley> work fine, we use Intel P3700 400GB for OSD Logs
[19:30] <burley> journals, rather
[19:31] <burley> and the 2 internal bays for the OS
[19:31] <burley> and had them throw in LSI 9207-8i controllers instead of the PERCs
[19:32] <burley> which I assume is still an available option
[19:34] * mattrich (~Adium@38.108.161.133) Quit (Quit: Leaving.)
[19:36] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[19:36] <cholcombe> gotcha.
[19:37] * mattrich (~Adium@38.108.161.133) has joined #ceph
[19:37] <cholcombe> burley, oh those are intels pcix cards. Wow. nice choice.
[19:37] * kefu (~kefu@114.92.110.67) has joined #ceph
[19:39] <cholcombe> burley, do those pcix cards keep the spinning drives pinned when the write load is heavy?
[19:39] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) Quit (Remote host closed the connection)
[19:39] <burley> in normal usage, the NVMe drive is totally and entirely bored
[19:39] <cholcombe> i'll bet
[19:39] <cholcombe> those things can write at like 2.5GB/s or more
[19:39] <burley> it gets pretty saturated during backfills
[19:40] <cholcombe> how's the ram usage during backfills?
[19:40] <burley> 1:12 is a pretty good ratio if you can accept the failure count
[19:40] <cholcombe> sure
[19:40] <burley> RAM usage hasn't been an issue
[19:40] <cholcombe> burley, there's actually a blog post about getting your stuff back after a journal failure
[19:41] <burley> we deployed initially with 32GB but decided to bump it
[19:41] <cholcombe> yeah you have 3GB/OSD about
[19:41] <cholcombe> that's plenty
[19:41] <monsted> you could use the intel 750 drive today - 90% of the speed at a quarter of the cost
[19:41] <burley> we also had the sticks laying around so it didn't cost anything
[19:41] <cholcombe> so i take it the 1GB/osd recommendation was too light?
[19:41] * reed (~reed@75-101-54-131.dsl.static.fusionbroadband.com) Quit (Quit: Ex-Chat)
[19:42] * reed (~reed@75-101-54-131.dsl.static.fusionbroadband.com) has joined #ceph
[19:42] <TheSov> so i know this is a math question but when you increase an object by 100 percent, to get another 100 percent relative to that you have to square it, correct?
[19:43] <monsted> TheSov: something + 100% is 2xsomething
[19:43] <monsted> not something^2
[19:43] <TheSov> right but i mean relative to the 2nd thing
[19:43] <TheSov> so i got thing 1, + 100% = thing 2 then i want to have +100 percent to thing 2
[19:43] <burley> The Intel 750 only has 70GB/day endurance
[19:44] <cholcombe> burley, i see. That's not bad but if you have a really heavy write workload that might not do
[19:45] <monsted> TheSov: that makes thing 2 x4, but it's just the multiplier that gets squared
[19:45] <monsted> (100+100%)+100% is 400, not 10000
[19:45] <TheSov> ok 4x thats what i wanted
[19:45] <TheSov> wait if i wanted 100 percent relative to that
[19:46] <TheSov> then it would be 8x
[19:46] <TheSov> are you sure we arent squaring here?
[19:46] <monsted> 8 isn't squared anything
[19:46] <TheSov> ahh right
[19:47] <monsted> (well, it's 2.something squared)
[19:47] <TheSov> wait i got now
[19:47] <TheSov> its binary multiplication
[19:47] <TheSov> ok now it makes sense
[19:47] <TheSov> thanks guys!
[19:47] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[19:47] <monsted> it's a binary shift, in principle :)
[19:48] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) has joined #ceph
[19:48] <rkeene> Well, in principle it's a power-of-two
[19:49] <monsted> exponential growth
[19:49] <TheSov> stop confusing me, i hate math!
[19:49] * sleinen1 (~Adium@2001:620:0:82::101) Quit (Ping timeout: 480 seconds)
[19:49] <monsted> math hates you too!
[19:49] <TheSov> well thats not true
[19:49] <TheSov> im just not that great at it
[19:49] <TheSov> i blame my teachers
[19:50] <TheSov> because i love watching numberphile
[19:50] <monsted> i blame the student :)
[19:50] <rkeene> Here, plot this out
[19:50] <rkeene> y=((sin(x*6))*((mod(int(abs(x)*10),2)*2)-1)*(((int(sqrt((abs((x-(0+(16/2)))/2)/((16/2)/2))/((abs((x-(0+(16/2)))/2)/((16/2)/2))+1))+0.3))-1)*-1))+((sin((x-90)*6))*((mod(int(abs(x)*10),2)*2)-1)*(((int(sqrt((abs((x-(45+(15/2)))/2)/((15/2)/2))/((abs((x-(45+(15/2)))/2)/((15/2)/2))+1))+0.3))-1)*-1))+(((mod(int(abs(x)*10),2)*2)-1)*(((int(sqrt((abs((x-(15+(5/2)))/2)/((5/2)/2))/((abs((x-(15+(5/2)))/2)/((5/2)/2))+1))+0.3))-1)*-1))+(((mod(int(abs(x)*10),2)*2)-1)*(((int(sqrt((abs((
[19:50] <TheSov> rkeene, i will murder you in your sleep
[19:50] <rkeene> Ha! I don't sleep
[19:50] <TheSov> fine be awake then :D
[19:51] <championofcyrodi> mod?
[19:51] <TheSov> modulo
[19:51] <TheSov> diveded by
[19:51] <TheSov> divided by*
[19:51] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[19:51] <rkeene> modulus operatio (the remainder of a division operation)
[19:51] <TheSov> the answer is the remainder
[19:51] <monsted> math is great! http://mathworld.wolfram.com/BatmanCurve.html
[19:51] <championofcyrodi> oh, like 3 % 2 in java...
[19:52] <TheSov> so 12 mod 7 is 5
[19:52] * kefu (~kefu@114.92.110.67) has joined #ceph
[19:52] <rkeene> championofcyrodi, Yeah, my calculator just didn't have a mod operator, so I had to implement it as a function
[19:52] <TheSov> championofcyrodi, yep exactly
[19:52] * dgbaley27 (~matt@c-67-176-93-83.hsd1.co.comcast.net) has joined #ceph
[19:53] <rkeene> My equation won't fit into wolfram's thingy
[19:53] <championofcyrodi> the last time i used it, was for monitoring the size of a file being written to HDFS... where the file's block size was reported as number of bytes, written until it was complete, at which point the block size would change to the HDFS block size, (64MB) in my case...
[19:53] * linjan__ (~linjan@176.195.14.90) has joined #ceph
[19:53] <championofcyrodi> so w/ my monitor, i would % by the block size, and if the remainder was not 0, then i knew it was still copying...
[19:53] <championofcyrodi> w/ a back off strategy.
[19:54] <rkeene> I write a huge amount of software, so I use it all the time
[19:54] <championofcyrodi> it was highly unlikely for anything to be exactly, 1024*1024*1024*64 bytes
[19:54] * Spikey (~Deiz@9S0AAC8US.tor-irc.dnsbl.oftc.net) Quit ()
[19:54] <championofcyrodi> and if it actually was... the back off strategy just assumed... "Well, the remainder has been zero over the last couple of checks, so it's probably done.'
[19:55] <championofcyrodi> not very elegant... but simple enough and worked
[19:55] <championofcyrodi> that was w/ hadoop 0.19... im sure 2.7 has some better api calls available now :)
[19:56] <championofcyrodi> and by hadoop, i just mean HDFS
[19:56] <championofcyrodi> (not the m/r portion)
[19:56] <TheSov> this may be a sensitive subject and all but why is sheepdog faster than ceph, like ceph is way more mature
[19:59] <championofcyrodi> just reading up on it... in my opinion, the slowest part of ceph is the idea a client has to hash out the location of the objects.... which i assume is a function of the cluster size affecting the cost per calculation.
[19:59] <rkeene> Probably a lot of reasons -- I investigated the slowness I found on my system and found that it was caused by writing to the journal opened O_SYNC and locking while doing so
[19:59] * kmARC_ (~kmARC@188-143-103-152.pool.digikabel.hu) has joined #ceph
[20:00] * kmARC (~kmARC@188-143-103-152.pool.digikabel.hu) has joined #ceph
[20:00] <championofcyrodi> but maybe not...
[20:01] <TheSov> the hashing is really fast i dont think thats it, and scaleio does something similar
[20:01] <TheSov> with scaleio i know tha emc's software uses the disk raw
[20:02] <TheSov> i have to look into sheepdog's works
[20:02] <rkeene> FWIW, I'm writing my own filesystem to deal with it (and the lack of on-read-checksumming) rather than trying to make Ceph faster
[20:03] <TheSov> rkeene, holy shit dude
[20:03] * linjan__ (~linjan@176.195.14.90) Quit (Ping timeout: 480 seconds)
[20:03] <TheSov> thats a helluva step
[20:03] <emre> anyone know anything about ceph-mesos/docker-ceph besides leseb_ who seems to be afk
[20:04] <TheSov> damn i a storage admin not a programmer!
[20:04] <TheSov> i cant do that stuff
[20:04] <rkeene> TheSov, I couldn't find any good alternatives... MooseFS looked good but didn't do the checksumming I wanted... I could run it on BtrFS but it still reportedly does not sync at the appropriate times
[20:04] <TheSov> on read hashing would be incredibly slow though
[20:04] * nsoffer (~nsoffer@109.65.255.15) has joined #ceph
[20:04] <rkeene> On read hashing is done in BtrFS, so not that slow
[20:05] <TheSov> rkeene, seperate the layers
[20:05] <rkeene> (And ZFS)
[20:05] <TheSov> thats what i am going to do
[20:05] * nsoffer (~nsoffer@109.65.255.15) Quit ()
[20:05] <TheSov> im going to build zfs boxes and use an OSD server to connect to them via iSCSI
[20:05] <rkeene> Basically this will be BtrFS that uses network I/O to talk to the disks rather than direct I/O
[20:05] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) has joined #ceph
[20:05] <rkeene> (Except slightly simpler, but still using a B-tree to deal with files)
[20:06] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) Quit ()
[20:06] <rkeene> 14-bit btree for links
[20:06] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) has joined #ceph
[20:09] * adrian15b (~kvirc@btactic.ddns.jazztel.es) Quit (Ping timeout: 480 seconds)
[20:10] <rkeene> My "monitors" will also be stateless (it has 3 components: 1. Monitors (stateless, peer-to-peer, k-complete); 2. Tree Nodes (which own the metadata for a given directory); and 3. Data Nodes (which own the contents of individual blocks/records) -- all blocks/records are identified by their SHA256 (chosen over SHA1 for better alignment, could also use SHA1 and padding) and the Trees really just map the tree to a record/block (they only exist to make sure that a pointer to
[20:10] <rkeene> The monitors are basically done since they are just passing around messages and pointers
[20:12] <rkeene> This makes snapshotting (on the directory level, or file level, but not filesystem level) easy to do atomically -- just copy the record ID of the current tree and save it
[20:13] * mattrich1 (~Adium@2600:1010:b018:dfee:adb1:a94d:6860:e6a4) has joined #ceph
[20:14] <TheSov> hmmm
[20:14] <TheSov> did any of you guys catch cds jewel?
[20:14] <TheSov> i been listenting to them on youtube
[20:14] <TheSov> some fascinating stuff
[20:15] <TheSov> like building the fsck module for cephfs
[20:15] <monsted> TheSov: why on earth would you separaate the OSD from ZFS with iscsi?
[20:16] <TheSov> so the osd's are working on zfs
[20:16] <monsted> why not just run the OSD on ZFS?
[20:16] <TheSov> because ZOL is worse than shit
[20:16] <ircolle> TheSov - ouch
[20:16] * evilrob00 (~evilrob00@cpe-72-179-3-209.austin.res.rr.com) Quit (Read error: No route to host)
[20:16] <TheSov> sorry but its true
[20:16] <monsted> nah, it's surprisingly good considering how horrible the rest of linux is ;)
[20:17] * ircolle is glad behlendorf isn't here
[20:17] <TheSov> i made a comparison about zol a while ago im tryint to remember it
[20:17] <rkeene> I have only really used ZFS on Solaris, and it took a long time for it to get stable. Most of my kernel panics were from ZFS.
[20:18] <monsted> ZFS on FreeBSD is rock solid, too
[20:18] <monsted> and it's getting really close on linux
[20:18] * fred`` (fred@earthli.ng) has joined #ceph
[20:18] <rkeene> ZFS on Solaris is.. solid-ish
[20:18] <rkeene> I only lost data with ZFS once
[20:18] <rkeene> (Due to a bug in ZFS)
[20:19] * Sysadmin88 (~IceChat77@2.125.96.238) has joined #ceph
[20:19] <monsted> i used ZFS on FreeBSD during the days of weekly or daily kernel panics, but never actually lost anything
[20:19] <TheSov> i said something to effect that ZFS was a brilliant masterpeice of art and science, and ZOL is taking that masterpeice smashing it with a hammer and putting it back together somewhere else with bluetack
[20:19] * mattrich2 (~Adium@38.108.161.133) has joined #ceph
[20:19] <TheSov> yes i beleive that was my quote
[20:19] * mattrich (~Adium@38.108.161.133) Quit (Read error: Connection reset by peer)
[20:20] * mattrich2 (~Adium@38.108.161.133) Quit ()
[20:20] * mattrich1 (~Adium@2600:1010:b018:dfee:adb1:a94d:6860:e6a4) Quit (Read error: Connection reset by peer)
[20:20] <snakamoto> Anyone here have JunOS MC-LAG configured to the server level?
[20:20] * thansen (~thansen@17.253.sfcn.org) has joined #ceph
[20:21] <monsted> TheSov: they're really only replacing the OS specific shim - replacing solaris or bsd specifics with linux. the bulk of openzfs is untouched.
[20:21] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[20:27] * evilrob00 (~evilrob00@cpe-72-179-3-209.austin.res.rr.com) has joined #ceph
[20:39] * kefu (~kefu@114.92.110.67) Quit (Max SendQ exceeded)
[20:43] * kefu (~kefu@114.92.110.67) has joined #ceph
[20:46] * kefu (~kefu@114.92.110.67) Quit ()
[20:47] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) Quit (Quit: bye!)
[20:48] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) has joined #ceph
[20:48] <TheSov> monsted, then why do i get such a huge performance difference?
[20:48] <TheSov> bsd with zfs, run disk tests, answer A. same system reinstall linux with ZOL, disk tests = B... and b sucks
[20:49] <monsted> ideally, we'd have the OSD software run on BSD
[20:49] <monsted> (or, preferably, all of ceph)
[20:49] <TheSov> yes
[20:49] <TheSov> ehhh
[20:49] <TheSov> osd
[20:50] <rkeene> Not a use case for me, otherwise I'd port it :-D
[20:51] <monsted> i have no idea how they managed to write it non-portable.
[20:51] <rkeene> It uses libaio, for one thing
[20:53] * neurodrone (~neurodron@108.60.145.130) has joined #ceph
[20:54] * QuantumBeep (~ylmson@ns316491.ip-37-187-129.eu) has joined #ceph
[20:56] * kmARC (~kmARC@188-143-103-152.pool.digikabel.hu) Quit (Ping timeout: 480 seconds)
[20:57] * kmARC_ (~kmARC@188-143-103-152.pool.digikabel.hu) Quit (Ping timeout: 480 seconds)
[20:57] <TheSov> aio?
[20:57] <TheSov> there is no aio for bsd?
[20:58] * nsoffer (~nsoffer@bzq-109-65-255-15.red.bezeqint.net) Quit (Ping timeout: 480 seconds)
[21:00] * dgbaley27 (~matt@c-67-176-93-83.hsd1.co.comcast.net) Quit (Quit: Leaving.)
[21:00] <[arx]> TheSov: http://sprunge.us/BGWE
[21:01] <TheSov> ok so whats the problem :D
[21:01] <TheSov> gawd i wish i knew programming
[21:03] * kefu (~kefu@114.92.110.67) has joined #ceph
[21:05] <monsted> linux aio was probably written to be similar, but incompatible, to aio on everything else :)
[21:08] * dopesong (~dopesong@78-60-74-130.static.zebra.lt) has joined #ceph
[21:10] * rendar (~I@host238-62-dynamic.247-95-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[21:13] * rendar (~I@host238-62-dynamic.247-95-r.retail.telecomitalia.it) has joined #ceph
[21:14] <championofcyrodi> TheSov: on the note of making things 'faster'... have you looked at HDFS' short circuit feature?
[21:14] <TheSov> ceph on hadoop?
[21:14] <championofcyrodi> no...
[21:15] <TheSov> hdfs is hadoop fs
[21:15] <championofcyrodi> the concept is, that if a client call is sourced from the same node in which the datanode resides (OSDs in our case), then TCP is 'short-circuited' and the data is accessed via unix socket.
[21:15] * Ceph-Log-Bot (~logstash@185.66.248.215) has joined #ceph
[21:15] * Ceph-Log-Bot (~logstash@185.66.248.215) Quit (Read error: Connection reset by peer)
[21:15] <championofcyrodi> which as we know, is FSB speed rather than network bound.
[21:15] <TheSov> right
[21:16] <championofcyrodi> for example, in my dev env... my guys use HDFS a lot.
[21:16] <championofcyrodi> by using a sequentially faster filesystem like, BTRFS... with short circuiting...
[21:16] <championofcyrodi> we achieve read speeds of native disk
[21:17] <TheSov> i am confuse
[21:17] <championofcyrodi> of course, if a block/ob is located on another node, then it utilizes network bandwidth...
[21:17] <TheSov> osd can use a socket connection to disk?
[21:17] <championofcyrodi> i don't believe it can...
[21:17] <championofcyrodi> ceph osd ~ hdfs datanode
[21:17] <championofcyrodi> they are not really that similar, except in general concept.
[21:18] <championofcyrodi> ceph monitor ~ hdfs namenode
[21:18] <TheSov> I liken osds to be more like torrenters
[21:18] <championofcyrodi> yes...
[21:18] <TheSov> and everyone is a seed!
[21:18] <TheSov> sort of
[21:18] <championofcyrodi> that is a good way to describe...
[21:18] <championofcyrodi> now imagine this...
[21:18] <championofcyrodi> your cluster is 3 nodes.
[21:18] <TheSov> ok
[21:18] <championofcyrodi> replica of say... 1
[21:19] <championofcyrodi> thus 1/3 of your file should be on a node.
[21:19] <TheSov> with you
[21:19] <championofcyrodi> if your client call originates from the 1 node... you get 1/3 of the data at native speed
[21:19] <championofcyrodi> and 2/3 via network.
[21:19] <TheSov> sounds good
[21:19] <championofcyrodi> down side...
[21:19] <championofcyrodi> client code has to share the same node.
[21:19] <TheSov> agree
[21:20] <championofcyrodi> this works well with hadoop because mapreduce nodes can co-exist w/ datanodes.
[21:21] <TheSov> ok
[21:22] <TheSov> but who would want to run code on the storage units? people usually want them dedicated
[21:22] <TheSov> When i build computer I am not going to pack it full of disk and when i build storage i am not going to pack it full of cpu
[21:23] <TheSov> only the bare essentials
[21:23] * adrian15b (~kvirc@218.Red-83-51-82.dynamicIP.rima-tde.net) has joined #ceph
[21:24] * QuantumBeep (~ylmson@7R2AADJ51.tor-irc.dnsbl.oftc.net) Quit ()
[21:24] * Nijikokun (~sixofour@freedom.ip-eend.nl) has joined #ceph
[21:25] <koollman> TheSov: that's pretty much the point of hadoop. bringing the code to the data, rather than the data to the code
[21:25] <championofcyrodi> ^this
[21:25] <championofcyrodi> code is smaller
[21:25] <koollman> and data is 'big' :)
[21:26] <TheSov> koollman, have you seen the research on running hadoop over ceph?
[21:26] <championofcyrodi> in reality... TheSov is right though...
[21:26] <TheSov> it runs almost 20 percent faster
[21:26] <TheSov> why is that you think?
[21:26] <championofcyrodi> you do need CPU+RAM to run code effectively.
[21:26] <championofcyrodi> and that drives up the cost of a storage node...
[21:26] <championofcyrodi> to where it's not really a storage node anymore.
[21:26] <championofcyrodi> as my boss says, "There is no free lunch."
[21:27] <TheSov> if you google hadoop over ceph you will see many articles talking about why using ceph instead of hdfs is superior
[21:27] <championofcyrodi> except when you have clients who have applications that only run on hadoop.
[21:27] <TheSov> as my former employer used to say "you dont ask the mechanic to milk the cows unless you want greasy milk"
[21:28] <championofcyrodi> lmao
[21:28] <TheSov> i dont know exactly what he meant but i take it, thats bad
[21:33] * adrian15b (~kvirc@218.Red-83-51-82.dynamicIP.rima-tde.net) Quit (Ping timeout: 480 seconds)
[21:38] * linjan__ (~linjan@176.195.14.90) has joined #ceph
[21:40] <rkeene> championofcyrodi, It's TANSTAFL (tan-staffl)
[21:42] <championofcyrodi> rkeene: just what i need... another acronym
[21:42] <championofcyrodi> ;)
[21:43] <rkeene> JWINAA
[21:46] * kefu (~kefu@114.92.110.67) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[21:52] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[21:54] * Nijikokun (~sixofour@9S0AAC8ZH.tor-irc.dnsbl.oftc.net) Quit ()
[21:54] * KrimZon (~Guest1390@192.42.116.16) has joined #ceph
[21:59] * kevinc (~kevinc__@client64-228.sdsc.edu) has joined #ceph
[22:02] * linjan__ (~linjan@176.195.14.90) Quit (Ping timeout: 480 seconds)
[22:07] * fdmanana (~fdmanana@bl13-145-124.dsl.telepac.pt) Quit (Ping timeout: 480 seconds)
[22:10] <neurodrone> Is there a place where can I find some usage examples for ceph-syn?
[22:11] <neurodrone> Is that even a recommended tool for simulating reads/writes on the cluster? I don???t want to bench them though so not trying to simulate load.
[22:12] <TheSov> linux needs a sqliosim
[22:12] <TheSov> things like DD only test linears
[22:13] <gleam> fio has an rbd driver
[22:14] <TheSov> my networking team thinks its ok to put ceph in a vlan by itself
[22:14] <TheSov> dear god help me
[22:14] <monsted> why wouldn't it be?
[22:14] <TheSov> dunno maybe the gigabits of data running through a router may be a bad idea
[22:15] <monsted> meh
[22:15] <monsted> routers can do terabits per second
[22:15] <TheSov> not the shitty corporate ciscos
[22:15] <TheSov> gawd i hate cisco
[22:16] <TheSov> everywhere i worked it was all ABC shops but now im at a cisco shop
[22:16] <rkeene> My Compute and Storage nodes are on different VLANs, they just have each other's VLANs tagged to the other
[22:16] <monsted> plenty of ciscos in the terabit range too
[22:16] <TheSov> yeh well here we dont do that, we route between vlans for security reasons
[22:17] <rkeene> I only have different VLANs for PXE-booting reasons (all my nodes PXE boot, except the PXE boot servers) -- most of my servers don't even have disks
[22:17] <TheSov> please tell me you have physically seperate storage and lan networks
[22:18] <rkeene> "LAN networks" ?
[22:18] <rkeene> You mean the user VLANs ?
[22:18] <TheSov> "LAN"
[22:18] <monsted> they're all LANs :)
[22:18] <TheSov> not V
[22:18] <TheSov> actual LANS
[22:18] <monsted> LANs are local, as opposed to WANs. it doesn't matter what's in them.
[22:19] <TheSov> "regular" access lan, all your normal shit. Storage, and backups
[22:19] <rkeene> I'm not sure why you think that: 1. The V is important/unimportant; and 2. Repeat networks (Local Area Network networks)
[22:19] <monsted> i bet TheSov is the kind of person who calls a storage array a SAN
[22:19] <TheSov> no...
[22:19] <TheSov> im using office nomenclature
[22:20] <TheSov> when in the office you say the LAN
[22:20] <monsted> those people need to be assassinated
[22:20] <TheSov> its all your normal network access
[22:20] <rkeene> I don't have an office, just a few datacenters
[22:20] <TheSov> storage network is different from that
[22:20] <monsted> the storage network is the SAN
[22:20] <TheSov> yeah
[22:20] <TheSov> exactly
[22:20] <monsted> (even if it's running on ethernet)
[22:21] <TheSov> and i prefer to have my physically isolated from the LAN
[22:21] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[22:21] <rkeene> User VLANs are on the other side of a Controlled Interface (CI) from the compute/storage VLANs
[22:22] <monsted> i wouldn't call a ceph network a SAN, though. too many servers and applications and such :)
[22:22] <TheSov> rkeene, have you used IFoIP?
[22:22] <monsted> not enough HDS arrays
[22:22] <TheSov> ceph isnt a san per se, but the "public network" side should be on the SAN
[22:22] <rkeene> monsted, USP-V 9990 here
[22:23] <rkeene> I have no idea what IFoIP is
[22:23] <TheSov> err backworks
[22:23] <TheSov> words
[22:23] <TheSov> IPofIF
[22:23] <TheSov> IPoIF
[22:23] <TheSov> damnit IB
[22:24] <TheSov> IPoIB
[22:24] <rkeene> Still no idea
[22:24] <TheSov> ip over infiniband
[22:24] <rkeene> Oh, yes, I've done IPoIB as well as IP over Fibrechannel
[22:24] <TheSov> do you get consistent performance on that?
[22:24] <rkeene> (Solaris sucks at fcip -- which is what they call IP over Fibrechannel)
[22:24] * KrimZon (~Guest1390@7R2AADJ8B.tor-irc.dnsbl.oftc.net) Quit ()
[22:24] <rkeene> It depends on the platform
[22:25] * CobraKhan007 (~skrblr@tor4.deklomp.se) has joined #ceph
[22:25] <monsted> FCIP is a protocol to move FC over IP :)
[22:25] <TheSov> i was trying to get my ceph test cluster to use IPoIB for its private network, and its....slow
[22:25] <monsted> (as is iFCP)
[22:25] <rkeene> monsted, Which is why it's annoying that Solaris calls IPoFC "fcip"
[22:25] <monsted> rkeene: solaris is generally annoying. happily, they're killing themselves slowly.
[22:26] <rkeene> ( http://docs.oracle.com/cd/E23823_01/html/816-5177/fcip-7d.html )
[22:26] <TheSov> monsted, theres a company out there still licensing and selling sco...
[22:26] <rkeene> For a long time talking fcip on Solaris just caused a kernel panic
[22:27] <monsted> TheSov: and yet, noone cares about them. i'm looking forward to oracle being in the same boat.
[22:27] <TheSov> you dont care but one of my customers a dentists office with 40 chairs, uses softech, a company who wants to install sco on a modern day machine
[22:27] <TheSov> i want to murder them
[22:27] <TheSov> slowly
[22:27] <TheSov> with a olive fork
[22:28] <TheSov> thats been worn to a nub
[22:29] <TheSov> sorry that was dark...
[22:31] <monsted> i generally opt for "fuck them with a cactus"
[22:31] <championofcyrodi> so is a rados gateway the only way to load balance ceph client calls?
[22:32] * adrian15b (~kvirc@218.Red-83-51-82.dynamicIP.rima-tde.net) has joined #ceph
[22:32] <monsted> ceph traffic is load balanced by default. what kind of clients do you mean?
[22:33] <championofcyrodi> well the ceph client needs to be able to TCP route to the ceph osds right?
[22:33] <monsted> yes
[22:33] * Nacer (~Nacer@37.163.21.213) Quit (Ping timeout: 480 seconds)
[22:33] <championofcyrodi> and you might not consider it secure to have 'clients' in the same subnet....
[22:33] <championofcyrodi> so how do you provide ceph storage to 'clients' w/o them sitting on your cluster?
[22:34] <monsted> the clients could be routed in.
[22:34] <championofcyrodi> could or have to be?
[22:34] <championofcyrodi> sounds to me like they have to be...
[22:34] <monsted> you can do pretty much anything you want
[22:34] <championofcyrodi> thus your router is always the bottle neck... and
[22:34] * Nacer (~Nacer@80.12.35.73) has joined #ceph
[22:34] <bearkitten> does rados cppool overwrite the target pool or just copy the contents for the source pool to the target pool?
[22:35] <championofcyrodi> now your osds have to have a route back out to the client to respond...
[22:35] <championofcyrodi> does not seem feasible for hundreds of nodes with WAN clients.
[22:35] <monsted> championofcyrodi: "routers" with plenty of bandwidth are cheap these days
[22:36] <TheSov> you guys have an odd idea of cheap
[22:36] <championofcyrodi> ^this
[22:36] <championofcyrodi> e.g. i hear VLANs suck w/ ceph because of routing bandwidth...
[22:36] <monsted> you could run the ceph nodes on a $5000 Juniper EX4300 and have four 40GigE uplinks to everything else
[22:37] <championofcyrodi> but then on the other hand, i hear that it's fine and just get a $5000 router.
[22:37] <championofcyrodi> lol
[22:37] <TheSov> 5k?
[22:37] <TheSov> thats like 72TB of storage
[22:37] <monsted> sorry, $3k
[22:37] <monsted> (for a 48 port)
[22:37] <championofcyrodi> but then you need 40GibE Nics for each node right?
[22:38] <TheSov> yep im right, for 10k i can get you 144tb of storage raw
[22:38] * dyasny (~dyasny@173.231.115.59) Quit (Ping timeout: 480 seconds)
[22:38] <championofcyrodi> TheSov: my thoughts stem from your statement about vlans... was thinking it might be better to use subnet masking and custom client routes.
[22:39] <monsted> i was thinking GigE to the host, room for 24 of them in a $1700 EX4300-24T, then you have 40G links to wherever your clients live (more EX4300s, maybe?) and the switches can do the routing.
[22:39] <TheSov> hmmm, well i tend to think of Ceph traffic like bittorrent traffic so id rather have it connected to Ceph Area Network!
[22:39] <monsted> s/host/node/
[22:40] <championofcyrodi> so your ceph might be 172.16.1.0/24, and clients be, 172.16.2.0/24... with rados gateways at 172.16.3.0.0/22.
[22:40] <championofcyrodi> sorry, radosgw at 172.16.0.0/22
[22:41] <monsted> that makes no sense.
[22:41] <championofcyrodi> then you can do Ceph<->gateway<->client on the same LAN
[22:41] <TheSov> well i currently use strictly RBD, so in my case they are all on 10.4.0.0 to 10.4.127.255.254
[22:41] <monsted> you can't have overlapping subnets
[22:41] <TheSov> well i currently use strictly RBD, so in my case they are all on 10.4.0.0 to 10.4.127.254
[22:42] <championofcyrodi> mosted: yes you can
[22:42] <TheSov> well it will be, right now we are in test
[22:42] <monsted> championofcyrodi: if .1.0/24, .2.0/24 and .0.0/22 are three different vlans, they wouldn't be able to speak to each other
[22:43] <championofcyrodi> they arent vlans.
[22:43] <monsted> (even with routers involved)
[22:43] <championofcyrodi> subnets
[22:43] <championofcyrodi> vlans use frame tagging.
[22:43] <monsted> thank you, i design datacenter networks for a living
[22:43] <TheSov> thats an interesting living
[22:43] <TheSov> just the networks?
[22:44] <magicrobotmonkey> CLOS or gtfo?
[22:44] <championofcyrodi> then why are you talking about VLANS when i'm talking about subnetting?
[22:44] <monsted> TheSov: and most of the rest of the datacenters, then i build them
[22:44] <monsted> championofcyrodi: if you're using different subnets, you usually have them in their own vlans, since you need routers between them anyway
[22:44] <TheSov> oh yeah? whats your opinion on raised flooring, i think its more of a gimmick than a necessity, but thats just my opinion
[22:45] <monsted> TheSov: i don't particularly like them, but if they're already in place i'll stick the cooling crap down there
[22:45] <TheSov> awesome
[22:46] <magicrobotmonkey> you don't *have* to have different subnets in different vlans
[22:46] <TheSov> yes they can be the same
[22:46] <TheSov> but routing between them is difficult :D
[22:46] <championofcyrodi> one of the first things traditionally taught with subnets, is how they can be used to provide security on a single LAN.
[22:46] <monsted> championofcyrodi: but if your rados gateways are in .0.0/22 and your ceph nodes are in .1.0/24, all in the same switched network (no routers), the nodes wouldn't be able to reach .0.*
[22:47] <championofcyrodi> by overlapping certain blocks of IPs.
[22:47] <monsted> noone teaches security by overlapping subnets
[22:48] <championofcyrodi> :::yawn:::
[22:48] <monsted> since it generally won't work.
[22:48] <magicrobotmonkey> subnets dont provide security
[22:48] <TheSov> oh gawd, its like im back in security+ training
[22:48] <magicrobotmonkey> lol
[22:48] <TheSov> can we stop swinging our dicks at each other, we are all gifted in our way
[22:48] * georgem (~Adium@206.108.127.16) has joined #ceph
[22:49] <TheSov> and i am awesome, even my mom thinks so
[22:49] <rkeene> I'm just normal
[22:49] <TheSov> actually you seem to be very on top of things
[22:49] <championofcyrodi> magicrobotmonkey: if you can't change the IP on a server with 192.168.0.0/24... how do you connect to a server with 10.10.10.23/24 ?
[22:50] <TheSov> do you happen to have OCD by chance?
[22:50] <rkeene> But I do a lot of things with networking -- I really like my packet-filtering in userspace project
[22:50] <rkeene> But it's over a decade old now :-(
[22:50] <championofcyrodi> its a rhetorical question, you cant.
[22:50] <rkeene> http://www.rkeene.org/viewer/tmp/pnfq-0.1.15/pnfq.c.htm
[22:50] <rkeene> TheSov, Nope nope nope
[22:50] <TheSov> did i just unknowingly send you my ssh keys?
[22:51] <TheSov> there seems to be a lot of "hax" on my screen after clicking that hah
[22:51] <championofcyrodi> https://uk.answers.yahoo.com/question/index?qid=20090122150154AADsdHw
[22:51] <rkeene> Unknowingly ? :-D
[22:51] <TheSov> https://blog.mozilla.org/security/2015/08/06/firefox-exploit-found-in-the-wild/
[22:51] <rkeene> I'm just saying that if you visit my web page, is it really unknowingly that it will go out and find your SSH key ?
[22:51] <championofcyrodi> you can read about how subnets are used for security... since it was done long before VLANS came along.
[22:52] <TheSov> but the problem of subnets without vlans is that i can always change my ip
[22:52] <rkeene> I my userspace packet filter (PNFQ) will let you run a Tcl script *PER PACKET*... I had it nmap OS-identify everyone who was talking to me and block Windows boxes that it identified in real-time
[22:52] <rkeene> I my -> I mean, my
[22:52] <monsted> championofcyrodi: yes, subnets segregate hosts. the point was that *overlapping* them doesn't work.
[22:53] <TheSov> rkeene, how much cpu does that guy use?
[22:53] * marrusl (~mark@cpe-67-247-9-253.nyc.res.rr.com) Quit (Quit: bye!)
[22:53] <TheSov> i imagine a script per packet basis would be quite intensive given the script
[22:53] <rkeene> TheSov, I wrote it about a decade ago and we were just about to upgrade to OC-192 from OC-48
[22:53] <monsted> championofcyrodi: the value of subnets without vlans is questionable, but it's better than nothing.
[22:54] <rkeene> TheSov, So to handle line-rate 10Gbps (OC-192 is ~9.9Gbps) I had an 8 core box
[22:54] <rkeene> I think 2.4GHz each core... this was a while ago
[22:54] <monsted> and you need something to route between them anyway, so it's going to pass through (probably) a L3 switch anyway
[22:54] * CobraKhan007 (~skrblr@7R2AADJ9E.tor-irc.dnsbl.oftc.net) Quit ()
[22:54] * Miho (~homosaur@mail.calyx.com) has joined #ceph
[22:55] <rkeene> TheSov, Tcl is pretty fast, it's JIT'd when you first run it and every other time just the bytecode is executed, we're talking microseconds per packet
[22:55] <rkeene> (Well, fractions of microseconds)
[22:55] <TheSov> hmm i always wanted a script that if someone ever portscanned me i would portscan them back and then flood udp traffic into any open ports i saw on the other side
[22:55] * adrian15b (~kvirc@218.Red-83-51-82.dynamicIP.rima-tde.net) Quit (Ping timeout: 480 seconds)
[22:56] <rkeene> Also, once it identified you it could add you to a list (implemented as binary tree, whose search time is in nanoseconds) so it trusted you/hated you more quickly
[22:56] <rkeene> That's basically what this did
[22:56] <monsted> TheSov: IDP solutions are commonly used to detect the port scan
[22:56] <TheSov> oddly enough my company portscans for a living
[22:57] <TheSov> or at least its a very small portion of our business
[22:57] <monsted> so it's YOUR fault!
[22:57] <rkeene> Except it didn't wait for you to portscan, if you sent a single packet it would portscan you to try to figure out what to do -- while it was in the process of portscanning it would let packets go back and forth (so the portscan could work) and only block at the end
[22:57] <TheSov> no we scan our customers' networks
[22:58] <monsted> yeah, we had customers request that for the stuff we hosted, without telling us about it
[22:58] <TheSov> you have probably heard of us
[22:58] * logan (~a@216.144.251.246) Quit (Ping timeout: 480 seconds)
[22:58] <TheSov> well monsted you probably know us by name then and most likely hate us :D
[22:58] <monsted> it was fun when one of them did it and i just dropped their entire ip block on our core routers :)
[22:59] <TheSov> cuz we get savvis shutdowns on our scans all the time
[22:59] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) Quit (Remote host closed the connection)
[23:00] <emre> anyone know anything about docker-ceph?
[23:00] <rkeene> TheSov, Portions of the US Military network will blacklist you for sending a single packet to a whole lot of IPs
[23:00] <TheSov> i was reading about docker monitros
[23:00] <TheSov> monitors*
[23:00] <TheSov> but docker for osd? WHY?!
[23:00] <emre> i want to know if it's worth the effort of spinning up 5 new CENTOS slaves in my cluster
[23:00] <rkeene> (That is, there's a whole lot of IP ranges that are honeypots, if you send a single packet to them you can't talk to the entire network until you stop talking for 24 hours)
[23:00] <TheSov> rkeene, my company or a small part of it is employed by the US for security
[23:00] * madkiss (~madkiss@2001:6f8:12c3:f00f:fd80:d2:b64a:124d) has joined #ceph
[23:01] <rkeene> TheSov, I set this up :-D
[23:01] <emre> rkeene - but if that's the case, couldn't an attacker map all your honeypot ips?
[23:01] <rkeene> emre, It would be EXTREMELY time consuming
[23:01] <emre> not with AWS :)
[23:02] <TheSov> rkeene, nice job man
[23:02] <rkeene> And there are other triggers beyond just that
[23:02] <rkeene> That all have the same result -- you can't talk to the entire network (/16 in this case) until you stop talking completely for 24 hours
[23:02] <monsted> emre: you can detect scans without the need for honeypots
[23:03] <rkeene> Yeah, if you talk to a lot of different IPs that do not have connections established with you (there's a rate) it will also block you
[23:03] * brad_mssw (~brad@66.129.88.50) Quit (Quit: Leaving)
[23:04] <monsted> stuff like Arbor Peakflow just gets a netflow feed from your core routers and will detech quite a few types of attacks... and possibly mitigate them automagically
[23:04] <monsted> detect, even
[23:04] <rkeene> I was using a Linux box (pair) inline at the edge with iptables
[23:05] * madkiss (~madkiss@2001:6f8:12c3:f00f:fd80:d2:b64a:124d) Quit ()
[23:07] <championofcyrodi> monsted: this is what i was trying to explain: http://pastebin.com/63xEKpba
[23:07] <championofcyrodi> this takes advantage of full fabric capacity of a switch
[23:08] <championofcyrodi> with vlans
[23:08] <championofcyrodi> or a physical router
[23:08] <championofcyrodi> but does require 'routing nodes' that have dual nics.
[23:08] <championofcyrodi> you wouldnt get 40GbE per connection, but you would have several connections at 1GbE.
[23:09] <championofcyrodi> for those of us w/o 3-5K for a router.
[23:09] * madkiss (~madkiss@2001:6f8:12c3:f00f:d84c:2ff7:661e:811c) has joined #ceph
[23:09] <championofcyrodi> and another sack of money for the 40Gbe Nics to handle multiple 1Gbps clients
[23:09] <monsted> with that setup, which host would be configured with the /27 on an interface?
[23:10] <TheSov> wait im building the opposite here
[23:10] <TheSov> im building a bunch of ceph osd nodes with 2 1 gig ports lacp'd
[23:10] <TheSov> and my clients have 10 gig nics
[23:10] <championofcyrodi> both NICs would require that subnet on the routing nodes.
[23:10] <TheSov> my backend network is infiniband
[23:11] <TheSov> or rather IPoIB
[23:11] <championofcyrodi> the routing nodes are essentially dual NIC'd and only passing traffic through.
[23:11] <monsted> you'd have the routing nodes configured with, say, .1/28 and .17/28, rather than /27, otherwise they'd be confused about what traffic to send out of which interfaces
[23:13] <monsted> or you'd need a bunch of manual routing entries. and the /27 doesn't serve any purpose at all.
[23:13] <TheSov> i figure since the clients are asking for traffic from multiple ceph osd's the client needs the faster nic
[23:13] <TheSov> and the osd's only need what the disks can put out
[23:13] <monsted> it works perfectly well with just the two /28s
[23:14] * ilken (ilk@2602:63:c2a2:af00:fda8:cd0c:bf79:773c) Quit (Ping timeout: 480 seconds)
[23:14] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[23:15] * ilken (ilk@2602:63:c2a2:af00:fda8:cd0c:bf79:773c) has joined #ceph
[23:15] <championofcyrodi> yea, i was over thinking it a bit... but that is the gist of what i was considering to be an alternative to using VLAN tagging if all your clients and osds are on the same physical switch(es)
[23:15] <championofcyrodi> which is common when you can't afford additional router and switches
[23:16] <championofcyrodi> with high throughput
[23:16] <monsted> that's fine. but the original problem was that you wanted to *overlap* the blocks, which screws up the routing tables of the hosts that are on the larger subnet
[23:16] <championofcyrodi> or rather, you can't convince the idiots with all the money running the biz to spend half as much on network hardware as they do compute/storage.
[23:16] <TheSov> championofcyrodi, the terrible part is my company can afford it, they just dont think they should spend so much on storage
[23:17] <TheSov> which is how i even found ceph
[23:17] <championofcyrodi> same here
[23:17] <rkeene> monsted, You can have as many routing tables as you want !
[23:17] <championofcyrodi> i was given 10K to proof of concept... got it all up and running... works great... then got complaints it wasnt fast enough.
[23:17] <championofcyrodi> then when i said it would cost more to upgrade the networking, got frowny faces...
[23:17] <monsted> rkeene: sure, but if they contain nonsense, the traffic goes the wrong way :)
[23:17] <TheSov> championofcyrodi, this company is ok with non redundant storage, they keep buying shelves of FC running raids on each shelf striping the data accross all shevles
[23:18] <TheSov> its scary
[23:18] * ilken (ilk@2602:63:c2a2:af00:fda8:cd0c:bf79:773c) Quit (Max SendQ exceeded)
[23:18] <TheSov> i keep waiting for the day a shelf dies and i get blamed
[23:18] <rkeene> What you need is the RAID allocation guide
[23:18] <championofcyrodi> management likes to put prototypes in production, and tell themselves they oversaw the implementation of a final product :(
[23:18] <TheSov> what is need is redundant storage
[23:18] <rkeene> http://rkeene.org/projects/info/wiki/221 (I call them Trays (T) instead of shelves)
[23:18] <TheSov> holy sheis rkeene that is nice
[23:18] <monsted> TheSov: our old EMC Clariion CX600s and 700s had a fun fault where sometimes a failure of disk #9 would also kill disk #15. we lost a few RAID5s on that account.
[23:19] <rkeene> Compute your MTBF and setup a clock until failure happens :-D
[23:19] <championofcyrodi> i have found i need big disks w/ replication for archival, and smaller disks w/ RAID 0 for operations.
[23:19] <TheSov> please dont make my fears worse, i keep waiting for the go ahead on ceph from the QA
[23:19] <championofcyrodi> then when an op disk dies... claim down time... replace the disk.. .the reload the data from archival
[23:19] <rkeene> MTBF_AGG = MTBF_DISK / (N x T - R - 1)
[23:20] * ilken (ilk@2602:63:c2a2:af00:211a:5f71:f7d1:7b28) has joined #ceph
[23:20] <rkeene> The PDF is much better formatted, http://www.rkeene.org/tmp/raid-allocation-guide.pdf
[23:20] <monsted> championofcyrodi: ew, raid0
[23:21] <TheSov> thanks rkeene, you just showed me a countdown to my firing
[23:21] <TheSov> thanks a lot...
[23:21] <championofcyrodi> monsted: can you think of a faster config?
[23:21] * madkiss (~madkiss@2001:6f8:12c3:f00f:d84c:2ff7:661e:811c) Quit (Ping timeout: 480 seconds)
[23:21] * ilken (ilk@2602:63:c2a2:af00:211a:5f71:f7d1:7b28) Quit (Max SendQ exceeded)
[23:21] <championofcyrodi> lets see... RAID1.. nope, RAID5, nope...
[23:21] <monsted> championofcyrodi: depending on your application? JBOD.
[23:21] <rkeene> TheSov, Hey, with 27 disks arranged that way the *WORST* aggregate MTBF is still 5.4 years
[23:21] <TheSov> i have 170+ disks now
[23:21] <rkeene> Well, that's more problematic
[23:21] <TheSov> hold 1 moment ill get the number
[23:22] <TheSov> 188 disks in the bigger FC cluster
[23:22] <rkeene> How are they arranged ?
[23:23] <championofcyrodi> jbod concats disks... where raid0 stripes across...
[23:23] <rkeene> A giant RAID0 across them all ? :-D
[23:23] <TheSov> tier 1 is raid 10
[23:23] <TheSov> tier 2 is 9 disk raid 5
[23:23] * ilken (~ilke@2602:63:c2a2:af00:211a:5f71:f7d1:7b28) has joined #ceph
[23:23] <monsted> championofcyrodi: many disks each doing their own thing independantly in parallel is, ideally, faster than a striped RAID where they have to cooperate
[23:23] <TheSov> and tier 3 is 6 disk raid 6
[23:23] <rkeene> RAID10s MTBF is harder to compute
[23:23] <TheSov> 23 disks in t1, 99 in t2, 66 in t3
[23:24] * xabner (~xabner@2607:f388:1090:0:bd7f:79ca:50eb:f08) Quit (Quit: Leaving...)
[23:24] <monsted> championofcyrodi: no, JBOD is not concatenated, they're just many independent disks with nothing gluing them together
[23:24] <championofcyrodi> monsted: i'm only running 1 process on the RAID0 config... so my iops are the same thing over and over.
[23:24] <TheSov> we are about to add a t2 shelf soon
[23:24] <TheSov> err tray
[23:24] <championofcyrodi> and it's 5 disks in a single server.
[23:24] * Miho (~homosaur@7R2AADKAF.tor-irc.dnsbl.oftc.net) Quit ()
[23:25] <championofcyrodi> i'm getting the max throughput on the PCI-e raid controller anyway so a faster disk config wouldnt make a diff.
[23:25] <monsted> (most raid controller makers are idiots and some may call something else JBOD even if it isn't)
[23:25] * Cybertinus (~Cybertinu@cybertinus.customer.cloud.nl) Quit (Remote host closed the connection)
[23:26] <rkeene> So on Tier 2, 11x9 (RAID5+0), MTBF_AGG = 1million / (99 - (2 - 1)) == 13 months
[23:26] <championofcyrodi> its friday... i'm going to go drink beer
[23:26] <championofcyrodi> later guys!
[23:27] * Cybertinus (~Cybertinu@cybertinus.customer.cloud.nl) has joined #ceph
[23:27] <rkeene> 1million hours, that is
[23:27] <rkeene> TheSov, Statistically you're due for a failure every 13 months on Tier 2
[23:28] <TheSov> LOL
[23:28] * kevinc (~kevinc__@client64-228.sdsc.edu) Quit (Quit: Leaving)
[23:28] <TheSov> oh god
[23:28] <rkeene> (Failure of the entire array)
[23:28] <TheSov> please dont say that
[23:28] <TheSov> ive only been here 5
[23:28] <rkeene> Has it been 13 months yet ? :-D
[23:28] * rendar (~I@host238-62-dynamic.247-95-r.retail.telecomitalia.it) Quit ()
[23:29] <TheSov> yeah the system is 2 years old
[23:29] * elder_ (~elder@apis-host139.dsl.visi.com) Quit (Ping timeout: 480 seconds)
[23:29] <rkeene> Tier 3 11x6 (RAID6+0, MTBF_AGG = 1million hours / (66 - (3 - 1)) == 21 months
[23:30] <TheSov> thats assuming spares are filling and ure occurs correct?
[23:30] <rkeene> That's assuming the failures are in the worst possible locations
[23:30] <rkeene> Losing 2 disks in the same RAID5
[23:30] <rkeene> Or 3 disks in the same RAID6
[23:31] * madkiss (~madkiss@2001:6f8:12c3:f00f:b95b:d455:7d9f:578c) has joined #ceph
[23:31] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[23:31] <monsted> i was on call the week we lost ten drives and our agreement with EMC was that they would call us when the fault appeared, when the were on site and started replacement and when the drive was back in sync. that week suuuuucked.
[23:31] <monsted> s/when the/when they/
[23:32] * segutier (~segutier@sfo-vpn1.shawnlower.net) Quit (Ping timeout: 480 seconds)
[23:32] <TheSov> wait
[23:32] <TheSov> with ceph
[23:33] <TheSov> what are the odds for something size 3
[23:33] <TheSov> same as raid 6?
[23:33] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:3128:969b:59c9:5d58) has joined #ceph
[23:34] <rkeene> For the worst case placement, yes -- if you lose all 3 copies of your object it's gone
[23:34] <TheSov> does ceph have a localized case for such an event
[23:34] <TheSov> lets say i lose 3 osd's
[23:34] <TheSov> and all of them had object 155 on them
[23:34] <TheSov> size 3
[23:34] <rkeene> Then you no longer have object 155
[23:35] <TheSov> right
[23:35] <TheSov> but what happens to the RBD
[23:35] <monsted> then that PG is dead
[23:35] <TheSov> does it drop?
[23:35] <TheSov> the RBD using that PG
[23:35] <rkeene> It will return an error when you try to read it, and probably write it :-D
[23:35] <rkeene> I think you have to manually intervene there to get even writing back
[23:36] <rkeene> But it's easy enough to test
[23:36] <TheSov> but only for that object right
[23:36] <TheSov> the cluster will function normally
[23:36] <TheSov> ?
[23:36] <TheSov> in other words will ceph freak out
[23:36] <monsted> Ceph, with patent-pending swiss cheese storage!
[23:36] <rkeene> Other PGs that haven't lost all their replicas will be accessible, yes
[23:36] <TheSov> ok
[23:36] <TheSov> phew
[23:36] <TheSov> i know when i lose a shelf/tray the storage is down
[23:37] <TheSov> all of it
[23:37] <monsted> debugging a "disk" where a small portion of blocks aren't working - that sounds like fun.
[23:37] <TheSov> i dont want that case in ceph
[23:37] <rkeene> Yeah, that's not the case with Ceph -- it's stripped "down" instead of "across"
[23:37] <monsted> TheSov: well, you would usually separate the data into racks (or one of the other groups) to make sure data gets stored in three different places
[23:37] <rkeene> (Refering to the picture in the PDF: http://www.rkeene.org/tmp/raid-allocation-guide.pdf )
[23:38] <TheSov> right but my initial cluster wont be big enough for that
[23:38] <TheSov> my initial cluster will be the size of the backups storage volume
[23:38] <TheSov> actually about 20% more
[23:38] <monsted> TheSov: it'll by default try to store data on three different hosts, too
[23:39] <TheSov> right
[23:39] * madkiss (~madkiss@2001:6f8:12c3:f00f:b95b:d455:7d9f:578c) Quit (Ping timeout: 480 seconds)
[23:39] <monsted> (the crushmap thing makes my brain hurt)
[23:39] * logan (~a@216.144.251.246) has joined #ceph
[23:40] * madkiss (~madkiss@2001:6f8:12c3:f00f:f9bf:2797:ed5c:27f2) has joined #ceph
[23:41] <TheSov> rkeene, my mistake our t2 is 6 disk raid 5
[23:41] <TheSov> t3 is 9 disk raid 6
[23:41] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:3128:969b:59c9:5d58) Quit (Ping timeout: 480 seconds)
[23:41] <TheSov> monsted, i too am confused by crushmaps of sizes bigger than rack
[23:42] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:3c78:8753:47ed:3d6) has joined #ceph
[23:42] <TheSov> all i know is it will attempt to keep at least 1 copy apart from another all the way down the failure domain
[23:44] * yghannam (~yghannam@0001f8aa.user.oftc.net) Quit (Quit: Leaving)
[23:48] * madkiss (~madkiss@2001:6f8:12c3:f00f:f9bf:2797:ed5c:27f2) Quit (Ping timeout: 480 seconds)
[23:49] <rkeene> Tier 2 17x6 (RAID6+0, MTBF_AGG = 1million hours / (102 - (3 - 1)) == 13 months
[23:49] <rkeene> Oh, 6 disk RAID5
[23:49] <rkeene> Tier 2 17x6 (RAID5+0, MTBF_AGG = 1million hours / (102 - (2 - 1)) == 13 months
[23:50] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:3c78:8753:47ed:3d6) Quit (Ping timeout: 480 seconds)
[23:51] <rkeene> Tier 3 7x9 (RAID6+0), MTBF_AGG = 1million hours / (63 - (3 - 1)) == 22 months
[23:52] <rkeene> So... you're due for a failure :-)
[23:53] <rkeene> I guess a more balanced formula would take into account the tray/disk ratios... Let me get that real quick
[23:55] * Peaced (~Epi@176.10.104.240) has joined #ceph
[23:55] * madkiss (~madkiss@2001:6f8:12c3:f00f:5803:8201:f292:eed7) has joined #ceph
[23:56] <TheSov> oh god
[23:56] <TheSov> rkeene, dealing with non redundant storage sucks
[23:58] * madkiss1 (~madkiss@ip5b418369.dynamic.kabel-deutschland.de) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.