#ceph IRC Log

Index

IRC Log for 2016-10-11

Timestamps are in GMT/BST.

[0:08] * datagutt (~theghost9@45.32.239.246) has joined #ceph
[0:11] * johnavp1989 (~jpetrini@8.39.115.8) Quit (Ping timeout: 480 seconds)
[0:13] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[0:14] * fsimonce (~simon@95.239.69.67) Quit (Remote host closed the connection)
[0:18] * wjw-freebsd2 (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[0:19] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:2038:c77a:4100:5349) Quit (Ping timeout: 480 seconds)
[0:27] * dscastro (~dscastro@181.166.94.84) Quit (Remote host closed the connection)
[0:38] * datagutt (~theghost9@45.32.239.246) Quit ()
[0:39] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[0:42] * wak-work (~wak-work@2620:15c:2c5:3:d5e:6789:2141:5d46) Quit (Remote host closed the connection)
[0:48] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[0:49] * bene2 (~bene@nat-pool-bos-t.redhat.com) Quit (Quit: Konversation terminated!)
[0:51] * hbogert (~Adium@ip54541f88.adsl-surfen.hetnet.nl) Quit (Quit: Leaving.)
[0:52] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) Quit (Quit: Leaving.)
[0:52] * ntpttr_ (~ntpttr@134.134.139.70) Quit (Remote host closed the connection)
[0:53] * ntpttr_ (~ntpttr@192.55.55.41) has joined #ceph
[0:53] * wak-work (~wak-work@2620:15c:2c5:3:58a2:eedc:3ac4:707e) has joined #ceph
[0:53] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) has joined #ceph
[0:53] <- *johnavp1989* To prove that you are human, please enter the result of 8+3
[0:58] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) has joined #ceph
[1:03] * mevabox (~mevabox@157-52-27-196.cpe.teksavvy.com) has joined #ceph
[1:09] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) Quit (Quit: Leaving.)
[1:10] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) has joined #ceph
[1:17] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[1:18] * wak-work (~wak-work@2620:15c:2c5:3:58a2:eedc:3ac4:707e) Quit (Quit: Leaving)
[1:18] * oms101 (~oms101@p20030057EA118F00C6D987FFFE4339A1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[1:19] * mevabox (~mevabox@157-52-27-196.cpe.teksavvy.com) Quit (Ping timeout: 480 seconds)
[1:20] * wak-work (~wak-work@2620:15c:2c5:3:58a2:eedc:3ac4:707e) has joined #ceph
[1:24] * hoonetorg (~hoonetorg@fh.fh-joanneum.at) Quit (Ping timeout: 480 seconds)
[1:27] * oms101 (~oms101@p20030057EA4F0700C6D987FFFE4339A1.dip0.t-ipconnect.de) has joined #ceph
[1:27] * xarses_ (~xarses@64.124.158.3) Quit (Ping timeout: 480 seconds)
[1:29] * nathani1 (~nathani@2607:f2f8:ac88::) Quit (Quit: WeeChat 1.4)
[1:34] * nathani (~nathani@2607:f2f8:ac88::) has joined #ceph
[1:35] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[1:41] * mevabox (~mevabox@157-52-27-196.cpe.teksavvy.com) has joined #ceph
[1:41] * mevabox (~mevabox@157-52-27-196.cpe.teksavvy.com) Quit ()
[1:46] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[1:48] * mhack (~mhack@24-151-36-149.dhcp.nwtn.ct.charter.com) Quit (Remote host closed the connection)
[1:49] * ntpttr_ (~ntpttr@192.55.55.41) Quit (Remote host closed the connection)
[2:00] * Concubidated (~cube@68.140.239.164) Quit (Quit: Leaving.)
[2:06] * ledgr (~ledgr@88-222-11-185.meganet.lt) has joined #ceph
[2:08] * xinli (~charleyst@32.97.110.55) Quit (Ping timeout: 480 seconds)
[2:14] * ledgr (~ledgr@88-222-11-185.meganet.lt) Quit (Ping timeout: 480 seconds)
[2:16] * kristen (~kristen@134.134.139.82) Quit (Quit: Leaving)
[2:19] * Concubidated (~cube@h4.246.129.40.static.ip.windstream.net) has joined #ceph
[2:24] * ira (~ira@c-24-34-255-34.hsd1.ma.comcast.net) Quit (Quit: Leaving)
[2:28] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[2:28] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[2:37] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[2:44] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Quit: Leaving.)
[2:44] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:45] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:45] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:45] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:46] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:46] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:46] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:47] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:47] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:48] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:48] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:49] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:49] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:49] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:50] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[2:50] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[2:58] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[3:11] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) Quit (Ping timeout: 480 seconds)
[3:14] * Unai (~Adium@192.77.237.216) has joined #ceph
[3:14] * KindOne (kindone@0001a7db.user.oftc.net) has joined #ceph
[3:21] * Jeffrey4l (~Jeffrey@110.252.73.52) has joined #ceph
[3:31] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) Quit (Quit: Leaving.)
[3:33] * georgem (~Adium@69-165-135-139.dsl.teksavvy.com) has joined #ceph
[3:35] * mevabox (~mevabox@157-52-27-196.cpe.teksavvy.com) has joined #ceph
[3:38] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[3:42] * georgem (~Adium@69-165-135-139.dsl.teksavvy.com) Quit (Quit: Leaving.)
[3:42] * georgem (~Adium@206.108.127.16) has joined #ceph
[3:44] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) has joined #ceph
[3:46] * davidzlap (~Adium@2605:e000:1313:8003:d512:b4b9:9e04:e792) Quit ()
[3:47] * Unai (~Adium@192.77.237.216) Quit (Read error: Connection reset by peer)
[3:48] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[3:49] * kefu (~kefu@114.92.125.128) has joined #ceph
[3:53] * Concubidated (~cube@h4.246.129.40.static.ip.windstream.net) Quit (Quit: Leaving.)
[3:53] * Unai1 (~Adium@192.77.237.216) has joined #ceph
[3:53] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) has joined #ceph
[3:53] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[3:54] * Jeffrey4l_ (~Jeffrey@120.11.30.55) has joined #ceph
[3:57] * kefu (~kefu@114.92.125.128) Quit (Max SendQ exceeded)
[3:59] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:01] * jfaj (~jan@p20030084AD2E91006AF728FFFE6777FF.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[4:01] * Jeffrey4l (~Jeffrey@110.252.73.52) Quit (Ping timeout: 480 seconds)
[4:02] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Quit: cyphase.com)
[4:03] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[4:06] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) Quit (Quit: doppelgrau)
[4:07] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[4:07] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:11] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[4:11] * jfaj (~jan@p20030084AD2DDF006AF728FFFE6777FF.dip0.t-ipconnect.de) has joined #ceph
[4:12] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:13] * kefu_ (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:13] * kefu (~kefu@li1456-173.members.linode.com) Quit (Read error: Connection reset by peer)
[4:13] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Quit: cyphase.com)
[4:14] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[4:17] * kefu_ (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[4:17] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:21] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[4:22] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:23] * Malcovent (~Sliker@108.61.122.214) has joined #ceph
[4:23] * KindOne_ (kindone@h159.149.29.71.dynamic.ip.windstream.net) has joined #ceph
[4:25] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Quit: cyphase.com)
[4:25] * cyphase (~cyphase@2601:640:c401:969a:78cc:a6b9:21d7:4f0e) has joined #ceph
[4:26] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[4:28] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:30] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[4:30] * KindOne_ is now known as KindOne
[4:37] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Quit: cyphase.com)
[4:37] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[4:39] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[4:39] * Unai1 (~Adium@192.77.237.216) Quit (Read error: Connection reset by peer)
[4:40] * kefu (~kefu@li1456-173.members.linode.com) Quit (Remote host closed the connection)
[4:41] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[4:44] * kefu (~kefu@li1456-173.members.linode.com) Quit ()
[4:46] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Quit: cyphase.com)
[4:46] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[4:53] * Malcovent (~Sliker@108.61.122.214) Quit ()
[4:56] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Quit: cyphase.com)
[4:56] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[5:04] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:05] * sage (~quassel@2607:f298:5:101d:f816:3eff:fe21:1966) Quit (Remote host closed the connection)
[5:06] * sage (~quassel@2607:f298:5:101d:f816:3eff:fe21:1966) has joined #ceph
[5:06] * mevabox (~mevabox@157-52-27-196.cpe.teksavvy.com) Quit (Ping timeout: 480 seconds)
[5:07] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[5:24] * sage__ (~quassel@64.111.99.127) has joined #ceph
[5:27] * sage (~quassel@2607:f298:5:101d:f816:3eff:fe21:1966) Quit (Read error: Connection reset by peer)
[5:28] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:30] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[5:30] * kefu (~kefu@114.92.125.128) has joined #ceph
[5:43] * Vacuum_ (~Vacuum@88.130.211.134) has joined #ceph
[5:49] * Vacuum__ (~Vacuum@88.130.215.214) Quit (Ping timeout: 480 seconds)
[5:56] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[6:07] * ledgr (~ledgr@88-222-11-185.meganet.lt) has joined #ceph
[6:11] * yanzheng (~zhyan@125.70.23.12) has joined #ceph
[6:11] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[6:11] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[6:12] * walcubi (~walcubi@p5795B634.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[6:13] * walcubi (~walcubi@p5795B011.dip0.t-ipconnect.de) has joined #ceph
[6:15] * ledgr (~ledgr@88-222-11-185.meganet.lt) Quit (Ping timeout: 480 seconds)
[6:15] * kefu (~kefu@114.92.125.128) Quit (Read error: Connection reset by peer)
[6:15] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[6:17] * Unai1 (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[6:17] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[6:20] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[6:20] * wiebalck_ (~wiebalck@AAnnecy-653-1-50-224.w90-41.abo.wanadoo.fr) has joined #ceph
[6:24] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[6:26] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[6:27] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[6:30] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[6:30] * Unai1 (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[6:30] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[6:31] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[6:33] * kefu (~kefu@li1456-173.members.linode.com) Quit (Max SendQ exceeded)
[6:34] * kefu (~kefu@li1456-173.members.linode.com) has joined #ceph
[6:37] * KindOne_ (kindone@h159.149.29.71.dynamic.ip.windstream.net) has joined #ceph
[6:43] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[6:44] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[6:44] * KindOne_ is now known as KindOne
[6:45] * ntpttr_ (~ntpttr@192.55.55.41) has joined #ceph
[6:56] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) has joined #ceph
[6:56] <- *johnavp1989* To prove that you are human, please enter the result of 8+3
[6:56] * TomasCZ (~TomasCZ@yes.tenlab.net) Quit (Quit: Leaving)
[6:57] * ntpttr_ (~ntpttr@192.55.55.41) Quit (Ping timeout: 480 seconds)
[6:59] <ivve> anyone knows if there is any good way to limit iops/bandwidth to either rbd image or a pool (either would be good)?
[7:00] <rkeene> Something LIKE that is planned for the next release, IIRC, but it won't be a hard limit instead it'll be shares
[7:03] <ivve> oh nice
[7:03] <ivve> but how do you mean shares
[7:04] * Concubidated (~cube@h4.246.129.40.static.ip.windstream.net) has joined #ceph
[7:05] <ivve> i mean today i can just limit to hardware which kinda separates it at a low level which isn't great
[7:05] * garphy is now known as garphy`aw
[7:06] <ivve> much rather have one large pool or pools over one set of hardware and limit it through the pools, or even better through the images themselves would be totally awesome
[7:09] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[7:12] * wiebalck_ (~wiebalck@AAnnecy-653-1-50-224.w90-41.abo.wanadoo.fr) Quit (Quit: wiebalck_)
[7:20] <iggy> guarantees vs hard limits (probably)
[7:26] * rcfighter (~ylmson@46.166.190.194) has joined #ceph
[7:29] * Unai1 (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[7:29] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[7:30] <ivve> sounds resonable
[7:30] * ircolle (~Adium@2601:285:201:633a:20a5:7715:7f11:49bb) Quit (Quit: Leaving.)
[7:31] <ivve> is there anywhere i can find a list of "features" being implemented in the next release?
[7:33] <iggy> that will most likely be decided at the next conf in Barcelona in a few weeks
[7:34] <iggy> https://www.openstack.org/software/roadmap/ goes into how the process works
[7:37] <ivve> cools
[7:37] * garphy`aw is now known as garphy
[7:39] * rinek (~o@62.109.134.112) Quit (Quit: ~)
[7:41] <ivve> anyone got good experience with building larger petabyte size clusters?
[7:45] * vimal (~vikumar@121.244.87.116) has joined #ceph
[7:49] * rinek (~o@62.109.134.112) has joined #ceph
[7:53] * ndru (~jawsome@00020819.user.oftc.net) Quit (Ping timeout: 480 seconds)
[7:56] * rcfighter (~ylmson@46.166.190.194) Quit ()
[7:56] * pdhange (~pdhange@61-69-103-54.mel.static-ipl.aapt.com.au) has joined #ceph
[7:57] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) Quit (Read error: Connection reset by peer)
[7:57] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) has joined #ceph
[8:06] <iggy> lol, totally in the wrong channel
[8:07] <iggy> ivve: ignore everything I said
[8:18] * jamespag` (~jamespage@culvain.gromper.net) Quit (Read error: No route to host)
[8:18] * jamespage (~jamespage@culvain.gromper.net) has joined #ceph
[8:20] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[8:20] * Unai1 (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[8:23] * gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) Quit (Quit: Textual IRC Client: www.textualapp.com)
[8:23] * jermudgeon (~jermudgeo@southend.mdu.whitestone.link) Quit (Quit: jermudgeon)
[8:26] * gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) has joined #ceph
[8:26] * ledgr (~ledgr@88-222-11-185.meganet.lt) has joined #ceph
[8:27] * pdhange (~pdhange@61-69-103-54.mel.static-ipl.aapt.com.au) Quit (Quit: Leaving)
[8:34] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) has joined #ceph
[8:39] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) Quit (Remote host closed the connection)
[8:39] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) has joined #ceph
[8:39] * ledgr (~ledgr@88-222-11-185.meganet.lt) Quit (Remote host closed the connection)
[8:45] * rraja (~rraja@125.16.34.66) has joined #ceph
[8:49] * nardial (~ls@p5DC07229.dip0.t-ipconnect.de) has joined #ceph
[8:56] * dgurtner (~dgurtner@176.35.230.73) has joined #ceph
[9:00] * gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) Quit (Quit: Textual IRC Client: www.textualapp.com)
[9:00] * tdb_ (~tdb@myrtle.kent.ac.uk) has joined #ceph
[9:02] * gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) has joined #ceph
[9:02] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[9:02] * tdb (~tdb@myrtle.kent.ac.uk) Quit (Ping timeout: 480 seconds)
[9:04] * garphy is now known as garphy`aw
[9:06] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) has joined #ceph
[9:06] * ade (~abradshaw@pool-22.254.176.62.dynamic.wobline-ip.de) has joined #ceph
[9:11] * Xmd (~Xmd@78.85.35.236) has joined #ceph
[9:14] * AlexeyAbashkin (~AlexeyAba@91.207.132.76) has joined #ceph
[9:14] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[9:14] <Xmd> Hi. I Get 10.2.3 for update 10.2.2 from repo from ftp://ftp.gwdg.de/pub/opensuse/repositories/filesystems:/ceph:/jewel/openSUSE_Leap_42.1/x86_64.
[9:15] <Xmd> after update ceph -v show what version is 10.2.2 ?? rpm include 10.2.2 ?? but header include info 10.2.3 ?
[9:17] <Xmd> file from rpm show 10.2.2
[9:18] <Xmd> ceph-osd-10.2.3+git.1475228057.755cf99-5.1.x86_64.rpm
[9:19] * analbeard (~shw@5.153.255.226) has joined #ceph
[9:19] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Read error: Connection reset by peer)
[9:20] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) has joined #ceph
[9:20] * Alexey_Abashkin (~AlexeyAba@91.207.132.76) Quit (Ping timeout: 480 seconds)
[9:29] * dgurtner (~dgurtner@176.35.230.73) Quit (Ping timeout: 480 seconds)
[9:33] * branto (~branto@transit-86-181-132-209.redhat.com) has joined #ceph
[9:33] * fsimonce (~simon@95.239.69.67) has joined #ceph
[9:37] * nhm (~nhm@c-50-171-139-246.hsd1.mn.comcast.net) Quit (Remote host closed the connection)
[9:37] * nhm (~nhm@c-50-171-139-246.hsd1.mn.comcast.net) has joined #ceph
[9:37] * ChanServ sets mode +o nhm
[9:39] * debian112 (~bcolbert@c-73-184-103-26.hsd1.ga.comcast.net) Quit (Ping timeout: 480 seconds)
[9:39] <FidoNet> morning
[9:40] <peetaur2> Mornin'
[9:41] <FidoNet> so ??? my continuing cephfs woes :)
[9:41] * minnesotags (~herbgarci@c-50-137-242-97.hsd1.mn.comcast.net) Quit (Ping timeout: 480 seconds)
[9:41] <FidoNet> overnight (and after a few reboots and re-syncing ntp) the auth errors disappeared (for now) and the mds are up:active ??? however .. I???m still seeing heartbeat errors in the logs
[9:41] <FidoNet> mds.beacon.mds02 _send skipping beacon, heartbeat map not healthy
[9:41] <FidoNet> any ideas?
[9:42] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[9:44] <Be-El> too much load on the mds host / overloaded network?
[9:44] <Be-El> did you tune the mds_cache_size setting for your workload?
[9:45] <Be-El> is the metadata pool fast enough / is the meta data access in the data pool (note the difference!) fast enough?
[9:45] * b0e (~aledermue@213.95.25.82) has joined #ceph
[9:50] * dgurtner (~dgurtner@94.126.212.170) has joined #ceph
[9:51] * minnesotags (~herbgarci@c-50-137-242-97.hsd1.mn.comcast.net) has joined #ceph
[9:53] * debian112 (~bcolbert@c-73-184-103-26.hsd1.ga.comcast.net) has joined #ceph
[9:54] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[9:55] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[9:56] <FidoNet> shouldn???t be ??? the network is 2 x 10 Gig running at less than 200Mbit currently ???. the mds are currently VMs on a proxmox cluster, but the node has less than 4 VMs each currently .. .all ???test??? kit .. but dual 2Ghz Xeons with 96GB ram / etc ???
[9:56] <FidoNet> network interfaces are all 10% or less
[9:56] <FidoNet> mds_cache_size no ??? not yet ??? suggestions for tuning?
[9:57] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[9:57] * adamcrume (~quassel@2601:647:cb01:f890:c136:33db:27c5:a2dc) Quit (Quit: No Ping reply in 180 seconds.)
[9:57] <FidoNet> actually we did to a degree and set it to 500000
[9:58] * markl (~mark@knm.org) Quit (Remote host closed the connection)
[9:58] * bitserker (~toni@2.152.12.64.dyn.user.ono.com) has joined #ceph
[9:58] <FidoNet> of course the mds VMs only have 2GB ??? doh
[9:59] <FidoNet> just reading this - http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-July/002768.html
[10:00] * DrewBeer (~DrewBeer@216.152.240.203) Quit (Remote host closed the connection)
[10:00] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) Quit (Quit: doppelgrau)
[10:00] * DrewBeer (~DrewBeer@216.152.240.203) has joined #ceph
[10:00] * ron-slc (~Ron@173-165-129-118-utah.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[10:01] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[10:01] <Be-El> FidoNet: mds needs ram, ram, ram, ram on top of it, and of cause ram
[10:01] <FidoNet> yeh just discovering that
[10:01] <FidoNet> had been trying to understand what the mds really did ??? slowly learning :)
[10:02] <Be-El> if you are running jewel you can get a glance of what's going on using 'ceph daemonperf mds.XYZ' on the mds host
[10:02] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Ping timeout: 480 seconds)
[10:02] <Be-El> not sure whether that command is also available in former releases
[10:02] * efirs (~firs@98.207.153.155) Quit (Ping timeout: 480 seconds)
[10:02] * vbellur (~vijay@71.234.224.255) Quit (Ping timeout: 480 seconds)
[10:03] * TMM (~hp@dhcp-077-248-009-229.chello.nl) Quit (Quit: Ex-Chat)
[10:04] <FidoNet> we???re running jewel so .. :)
[10:04] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) has joined #ceph
[10:04] * garphy`aw is now known as garphy
[10:04] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) Quit (Remote host closed the connection)
[10:04] * DanFoster (~Daniel@2a00:1ee0:3:1337:147e:cb72:9454:1467) has joined #ceph
[10:06] <Be-El> i would propose to run a tool like atop or nmon in one terminal, ceph daemonperf in another one and a third one for the actual work
[10:07] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[10:09] <FidoNet> sounds like I have my day planned out :)
[10:11] * sudocat (~dibarra@192.185.1.20) Quit (Ping timeout: 480 seconds)
[10:11] * derjohn_mob (~aj@46.189.28.41) has joined #ceph
[10:13] <Be-El> you can also use 'ceph daemon mds.XYZ ....' commands to inspect running ops. the mds is just another ceph client, so the same performance debugging options with respect to OSD access also apply
[10:15] * vbellur (~vijay@2601:18f:700:55b0:5e51:4fff:fee8:6a5c) has joined #ceph
[10:15] * wjw-freebsd2 (~wjw@smtp.digiware.nl) has joined #ceph
[10:15] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Read error: Connection reset by peer)
[10:16] * ron-slc (~Ron@173.165.129.118) has joined #ceph
[10:22] * kefu (~kefu@li1456-173.members.linode.com) Quit (Ping timeout: 480 seconds)
[10:24] * kefu (~kefu@li401-71.members.linode.com) has joined #ceph
[10:31] * natarej (~natarej@101.188.54.14) has joined #ceph
[10:32] * kefu (~kefu@li401-71.members.linode.com) Quit (Max SendQ exceeded)
[10:33] * kefu (~kefu@li401-71.members.linode.com) has joined #ceph
[10:36] * CustosLimen (~CustosLim@ns343343.ip-91-121-210.eu) Quit (charon.oftc.net helix.oftc.net)
[10:37] * cyphase (~cyphase@000134f2.user.oftc.net) has joined #ceph
[10:37] * adamcrume (~quassel@2601:647:cb01:f890:bcda:c738:479d:4b57) has joined #ceph
[10:37] * yanzheng (~zhyan@125.70.23.12) Quit (Quit: This computer has gone to sleep)
[10:38] * IcePic_ (~jj@2001:6b0:5:1688:d4a3:d779:daa0:6904) has joined #ceph
[10:39] * CustosLimen (~CustosLim@ns343343.ip-91-121-210.eu) has joined #ceph
[10:39] * IcePic (~jj@c66.it.su.se) Quit (Ping timeout: 480 seconds)
[10:40] * rendar (~I@host133-71-dynamic.171-212-r.retail.telecomitalia.it) has joined #ceph
[10:42] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[10:44] * adamcrume (~quassel@2601:647:cb01:f890:bcda:c738:479d:4b57) Quit (Quit: No Ping reply in 180 seconds.)
[10:45] * huats (~quassel@stuart.objectif-libre.com) Quit (Read error: Connection reset by peer)
[10:45] * huats (~quassel@stuart.objectif-libre.com) has joined #ceph
[10:45] * doppelgrau (~doppelgra@132.252.235.172) has joined #ceph
[10:47] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:a095:7e95:6611:bc7) has joined #ceph
[10:48] * loicd (~loicd@211.ip-167-114-243.eu) Quit (Quit: quit)
[10:48] * loicd (~loicd@211.ip-167-114-243.eu) has joined #ceph
[10:49] * cyphase (~cyphase@000134f2.user.oftc.net) Quit (Ping timeout: 480 seconds)
[10:51] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[10:51] * kefu (~kefu@li401-71.members.linode.com) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[10:52] * debian112 (~bcolbert@c-73-184-103-26.hsd1.ga.comcast.net) Quit (Ping timeout: 480 seconds)
[10:55] * IcePic_ is now known as IcePic
[11:00] * tdb (~tdb@129.12.3.176) has joined #ceph
[11:00] * tdb_ (~tdb@myrtle.kent.ac.uk) Quit (Read error: Connection reset by peer)
[11:02] * debian112 (~bcolbert@c-73-184-103-26.hsd1.ga.comcast.net) has joined #ceph
[11:04] * TMM (~hp@185.5.121.201) has joined #ceph
[11:07] * derjohn_mob (~aj@46.189.28.41) Quit (Ping timeout: 480 seconds)
[11:09] * Pettis (~Heliwr@tor-exit.squirrel.theremailer.net) has joined #ceph
[11:10] * evelu (~erwan@2a01:e34:eecb:7400:4eeb:42ff:fedc:8ac) has joined #ceph
[11:20] * hbogert (~Adium@095-097-133-138.static.chello.nl) has joined #ceph
[11:25] * ledgr (~ledgr@88-119-196-104.static.zebra.lt) has joined #ceph
[11:28] * hbogert1 (~Adium@95.170.93.16) has joined #ceph
[11:28] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:a095:7e95:6611:bc7) Quit (Ping timeout: 480 seconds)
[11:34] * hbogert (~Adium@095-097-133-138.static.chello.nl) Quit (Ping timeout: 480 seconds)
[11:39] * Pettis (~Heliwr@tor-exit.squirrel.theremailer.net) Quit ()
[11:44] * maybebuggy (~maybebugg@2a01:4f8:191:2350::2) has joined #ceph
[11:51] * derjohn_mob (~aj@46.189.28.87) has joined #ceph
[11:51] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Ping timeout: 480 seconds)
[12:00] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[12:02] * brannmar (~Arfed@108.61.123.80) has joined #ceph
[12:03] * cyphase (~cyphase@c-50-148-131-137.hsd1.ca.comcast.net) has joined #ceph
[12:07] * maybebuggy (~maybebugg@2a01:4f8:191:2350::2) Quit (Quit: Leaving)
[12:08] * derjohn_mob (~aj@46.189.28.87) Quit (Ping timeout: 480 seconds)
[12:08] * maybebuggy (~maybebugg@2a01:4f8:191:2350::2) has joined #ceph
[12:12] * etamponi (~etamponi@net-93-71-251-206.cust.vodafonedsl.it) has joined #ceph
[12:13] <etamponi> Hi :) is ceph.com down at the moment? I cannot connect and I get "bad gateway"
[12:25] <peetaur2> etamponi: http://www.downforeveryoneorjustme.com/http://ceph.com/
[12:25] <peetaur2> down for me too
[12:25] <hbogert1> yep, mighty irritating
[12:25] <etamponi> thanks
[12:26] <peetaur2> download.ceph.com and docs.ceph.com also down...pretty lame; they should have hosted it on ceph
[12:31] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[12:32] * brannmar (~Arfed@108.61.123.80) Quit ()
[12:32] <zokko> it ain't anything useful on docs.ceph though ;)
[12:36] * yankcrime (~yankcrime@185.43.216.241) has joined #ceph
[12:36] * wjw-freebsd2 (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[12:39] * peetaur2 (~peter@i4DF67CD2.pool.tripleplugandplay.com) Quit (Remote host closed the connection)
[12:39] * [0x4A6F]_ (~ident@p508CD48D.dip0.t-ipconnect.de) has joined #ceph
[12:40] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[12:42] * [0x4A6F] (~ident@0x4a6f.user.oftc.net) Quit (Ping timeout: 480 seconds)
[12:42] * [0x4A6F]_ is now known as [0x4A6F]
[12:44] <ivve> been down since at least 06.00 CET this morning
[12:56] * derjohn_mob (~aj@46.189.28.54) has joined #ceph
[12:59] * InIMoeK (~InIMoeK@105-183-045-062.dynamic.caiway.nl) has joined #ceph
[13:07] <etamponi> ouch... but now the message has changed (ERR_ADDRESS_UNREACHABLE instead of bad gateway), perhaps they're taking some action
[13:10] * salwasser (~Adium@2601:197:101:5cc1:1ec:bdfd:c527:1da2) has joined #ceph
[13:14] * dgurtner (~dgurtner@94.126.212.170) Quit (Ping timeout: 480 seconds)
[13:14] * salwasser (~Adium@2601:197:101:5cc1:1ec:bdfd:c527:1da2) Quit ()
[13:14] * rraja is now known as rraja|afk
[13:19] <etamponi> ceph.com is back, waiting for download.ceph.com
[13:20] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) Quit (Quit: Leaving)
[13:27] * DanFoster (~Daniel@2a00:1ee0:3:1337:147e:cb72:9454:1467) Quit (Quit: Leaving)
[13:28] * DanFoster (~Daniel@2a00:1ee0:3:1337:9:fb59:d2eb:719e) has joined #ceph
[13:29] * georgem (~Adium@2605:8d80:681:e122:7d47:8eed:c01a:c7e5) has joined #ceph
[13:30] * georgem (~Adium@2605:8d80:681:e122:7d47:8eed:c01a:c7e5) Quit ()
[13:30] * georgem (~Adium@206.108.127.16) has joined #ceph
[13:35] * etienneme (~arch@69.ip-167-114-227.eu) Quit (Ping timeout: 480 seconds)
[13:51] <s3an2> etamponi, still looks to be a problem here - also tracker.ceph.com is giving 502 from nginx
[13:52] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[13:52] <InIMoeK> still down here
[13:54] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:a095:7e95:6611:bc7) has joined #ceph
[13:55] <etamponi> yeah, ceph.com is down again too
[13:57] * wjw-freebsd2 (~wjw@smtp.digiware.nl) has joined #ceph
[13:57] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Read error: Connection reset by peer)
[14:04] * garphy is now known as garphy`aw
[14:04] * rraja|afk is now known as rraja
[14:10] * georgem (~Adium@206.108.127.16) Quit (Ping timeout: 480 seconds)
[14:12] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Ping timeout: 480 seconds)
[14:14] * ira (~ira@c-24-34-255-34.hsd1.ma.comcast.net) has joined #ceph
[14:15] * djidis__ (~Chaos_Lla@5.153.234.114) has joined #ceph
[14:19] * bniver (~bniver@nat-pool-bos-u.redhat.com) has joined #ceph
[14:19] * dgurtner (~dgurtner@94.126.212.170) has joined #ceph
[14:20] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[14:22] * etienneme (~arch@69.ip-167-114-227.eu) has joined #ceph
[14:25] * georgem (~Adium@206.108.127.16) has joined #ceph
[14:29] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[14:30] * bniver (~bniver@nat-pool-bos-u.redhat.com) Quit (Ping timeout: 480 seconds)
[14:30] * bniver (~bniver@nat-pool-bos-u.redhat.com) has joined #ceph
[14:32] * Kurt (~Adium@2001:628:1:5:5489:32f9:27e5:3f42) Quit (Quit: Leaving.)
[14:32] * dgurtner_ (~dgurtner@94.126.212.170) has joined #ceph
[14:32] * djidis__ (~Chaos_Lla@5.153.234.114) Quit (Ping timeout: 480 seconds)
[14:33] * garphy`aw is now known as garphy
[14:35] * bene2 (~bene@2601:193:4101:f410:ea2a:eaff:fe08:3c7a) has joined #ceph
[14:38] * dgurtner (~dgurtner@94.126.212.170) Quit (Ping timeout: 480 seconds)
[14:43] * mhack (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[14:43] * Kurt (~Adium@2001:628:1:5:5450:738e:2f55:70f5) has joined #ceph
[14:55] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[14:55] * mhack (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[14:58] * mevabox (~mevabox@157.52.27.196) has joined #ceph
[15:05] * Hungerhu (~hunger@prodevops.net) has joined #ceph
[15:06] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[15:07] * mevabox (~mevabox@157.52.27.196) Quit (Quit: Leaving)
[15:08] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[15:09] <thoht> docs.ceph.com is down ?
[15:09] <s3an2> yes
[15:09] <s3an2> http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/
[15:11] * Hunger (~hunger@prodevops.net) Quit (Ping timeout: 480 seconds)
[15:15] * mhack (~mhack@nat-pool-bos-u.redhat.com) has joined #ceph
[15:16] * brad_mssw (~brad@66.129.88.50) has joined #ceph
[15:18] <etienneme> thoht: you can still read the doc here https://github.com/ceph/ceph/tree/master/doc
[15:19] <thoht> before rebooting a node of ceph cluster (for instance for patching), do you isolate the device from the cluster ?
[15:19] <etienneme> no
[15:19] <thoht> what is the good practice ? for now, i just reboot each node one by one, ceph became HEALTH_ERR and has some pg stuck. ive to wait HEALTH_OK before going to next device to patch.
[15:20] <s3an2> thoht, I normally just set noout and stop the osd's before doing the OS reboot.
[15:20] <etienneme> When I have to reboot a host I set noout and it's better if you can stop ceph-all before on the osd
[15:20] <thoht> i do "reboot" so i think systemd is detaching osd before ?
[15:21] <thoht> i mean stoping them
[15:21] <thoht> what is the goal of noout ?
[15:21] <IcePic> to stop the rest of ceph to try to "repair" the lost osds when you reboot a machine, if it will come back in a minute
[15:22] <doppelgrau> thoht: in my experience manual stopping is a good thing, if a mon runs on the same node, else some goinf fown mesages might get lost
[15:22] <doppelgrau> thoht: noout: make sure that the cluster don't mark the osds as out => rebalancing if for some reasons the reboot take longer than down+out
[15:23] <s3an2> noout: an OSD marked as out means that it might be running but doesn???t actually receive any data since it???s not part of the CRUSH Map (opposite of being marked in). Thus the option noout prevents OSDs from being marked out of the cluster. (https://www.sebastien-han.fr/blog/2013/04/17/some-ceph-experiments/)
[15:23] <doppelgrau> but I increased the timeout so 20 minutes, so I can usually reboot without noout (which makes scripting easier, continue when health ok)
[15:23] <thoht> so you run " ceph osd set noout" and that's it ? it is a global command then ?
[15:23] <s3an2> yes
[15:24] <thoht> then when finish; unset
[15:24] <s3an2> yep
[15:24] <thoht> ok i just did it; health goes to HEALTH_WARN
[15:24] <thoht> and i can see the flat appearing
[15:24] <thoht> flag
[15:24] <s3an2> HEALTH_WARN is expected with noout flag set
[15:25] <thoht> so i can create an alias to .bashrc alias reboot='ceph osd set noout ; /sbin/reboot" and add in rc.local : ceph osd unset noout
[15:26] * webertrlz (~oftc-webi@191.185.28.83) has joined #ceph
[15:26] <thoht> does that make sense for somebody lazy as me ?
[15:27] <webertrlz> Hey guys, how can I resize or clean ceph osd journal?
[15:27] <s3an2> I guess you can, I normally leave noout set untill I have done all my host reboots, and I check the status of PG's rather than the cluster health status for when to move onto next host
[15:27] <webertrlz> my OSDs are all near full and I think the problem is the journal
[15:28] <s3an2> something like this - test "$(ceph pg stat | sed 's/^.*pgs://;s/active+clean.*//;s/ //')" -eq "$(ceph pg stat | sed 's/pgs.*//;s/^.*://;s/ //')" && ceph health | egrep -sq "HEALTH_OK|HEALTH_WARN"
[15:29] <s3an2> I normally use ansible for it - something like this http://pastebin.com/hGpjQLVL
[15:31] * Nicho1as (~nicho1as@00022427.user.oftc.net) has joined #ceph
[15:32] * ira is now known as ira_away
[15:34] * minnesotags (~herbgarci@c-50-137-242-97.hsd1.mn.comcast.net) Quit (Ping timeout: 480 seconds)
[15:34] <webertrlz> ceph website is out :(
[15:35] * nardial (~ls@p5DC07229.dip0.t-ipconnect.de) Quit (Quit: Leaving)
[15:36] * northrup (~northrup@173.14.101.193) has joined #ceph
[15:36] <northrup> apologies for being redundant, it's not set in the header or in an announce...
[15:36] <northrup> does anyone know WTH is up with ceph.com ?
[15:37] <s3an2> http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/
[15:37] * jermudgeon (~jermudgeo@southend.mdu.whitestone.link) has joined #ceph
[15:37] <northrup> thanks s3an2
[15:37] <northrup> also ... isn't that just damned awesome :(
[15:38] <webertrlz> totally
[15:38] <webertrlz> specially when I have all near full osds '^^
[15:39] * rotbeard (~redbeard@relay.innovo-cloud.de) has joined #ceph
[15:39] * Racpatel (~Racpatel@2601:87:3:31e3::34db) has joined #ceph
[15:41] * xinli (~charleyst@32.97.110.54) has joined #ceph
[15:42] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[15:44] * rraja (~rraja@125.16.34.66) Quit (Ping timeout: 480 seconds)
[15:45] * theancient (~jasonj@173-165-224-105-minnesota.hfc.comcastbusiness.net) Quit (Quit: Leaving.)
[15:45] * theancient (~jasonj@173-165-224-105-minnesota.hfc.comcastbusiness.net) has joined #ceph
[15:46] * northrup (~northrup@173.14.101.193) Quit (Ping timeout: 480 seconds)
[15:47] * jermudgeon (~jermudgeo@southend.mdu.whitestone.link) Quit (Quit: jermudgeon)
[15:48] * salwasser (~Adium@a72-246-0-10.deploy.akamaitechnologies.com) has joined #ceph
[15:52] * evelu (~erwan@2a01:e34:eecb:7400:4eeb:42ff:fedc:8ac) Quit (Ping timeout: 480 seconds)
[15:54] * mhack (~mhack@nat-pool-bos-u.redhat.com) Quit (Ping timeout: 480 seconds)
[16:01] <mistur> webertrlz: I don't think it's something related to journal
[16:01] <mistur> webertrlz: maybe your pg_num is too low and you don't have a good balance between OSD
[16:02] <webertrlz> mistur: I use ceph-fs for temporary files only, the OSDs are 20GB, and journal size is 10GB per OSD, that's why I think it might be the journal
[16:03] * diver (~diver@95.85.8.93) has joined #ceph
[16:03] <mistur> 5G should be enouph
[16:03] * t4nk690 (~oftc-webi@85.115.23.42) has joined #ceph
[16:03] <mistur> specialy for a 20GB OSD
[16:03] <mistur> you can check http://cephnotes.ksperis.com/blog/2015/02/23/get-the-number-of-placement-groups-per-osd
[16:03] <diver> hey. any issues with download.ceph.com?
[16:03] <mistur> diver: http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/
[16:03] * vbellur (~vijay@2601:18f:700:55b0:5e51:4fff:fee8:6a5c) Quit (Ping timeout: 480 seconds)
[16:03] <diver> thanks!
[16:04] * mhack (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[16:04] <webertrlz> I was just looking how to change journal size, I just found it. I will try it now
[16:04] <mistur> webertrlz: this script will tell you how are balance datas on your OSDs
[16:04] * jagardaniel1 (~oftc-webi@2001:9b0:109:103::b6) has joined #ceph
[16:04] <webertrlz> Oh great! Thanks for the link
[16:05] <s3an2> diver, http://eu.ceph.com/ may help you
[16:07] * evelu (~erwan@37.160.193.215) has joined #ceph
[16:09] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[16:12] * kefu (~kefu@114.92.125.128) has joined #ceph
[16:12] * xinli (~charleyst@32.97.110.54) Quit (Remote host closed the connection)
[16:12] * xinli (~charleyst@32.97.110.53) has joined #ceph
[16:13] * rotbeard (~redbeard@relay.innovo-cloud.de) Quit (Quit: Leaving)
[16:16] * Hungerhu is now known as Hunger
[16:16] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[16:16] <diver> s3an2: thanks, will try
[16:16] * mhack (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[16:17] * scuttle|afk is now known as scuttlemonkey
[16:18] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[16:20] <webertrlz> mistur: resizing journal was enough to solve my problem. Still, 35 out of 60GB used is wierd as I have less than 1GB of files in cephfs
[16:21] <mistur> webertrlz: there is a garbage collector
[16:22] <mistur> when you delete a file, object are not delete on time
[16:22] <mistur> there is a delay
[16:22] <webertrlz> Right.. and I create and delete about 2000 files per minute, so that might be the problem
[16:23] * sage__ is now known as sage
[16:23] <webertrlz> is there a way to force garbage collect?
[16:25] * northrup (~northrup@75-146-11-137-Nashville.hfc.comcastbusiness.net) has joined #ceph
[16:25] <mistur> webertrlz: mmh I don't know, in fact I have a gc for radosGW
[16:25] * vbellur (~vijay@nat-pool-bos-t.redhat.com) has joined #ceph
[16:25] <mistur> webertrlz: but I don't know for cephfs
[16:26] <webertrlz> mistur: I see. Thank you anyway
[16:26] <webertrlz> When ceph.com is back online I'll look in the docs
[16:26] <mistur> yup
[16:27] <webertrlz> maybe it can be a line in the conf file
[16:31] * t4nk690 (~oftc-webi@85.115.23.42) Quit (Quit: Page closed)
[16:36] * kristen (~kristen@134.134.139.74) has joined #ceph
[16:39] <s3an2> webertrlz, https://github.com/ceph/ceph/tree/master/doc
[16:41] * rotbeard (~redbeard@185.32.80.238) has joined #ceph
[16:41] * vata (~vata@207.96.182.162) has joined #ceph
[16:41] <webertrlz> s3an2: nice :D
[16:44] * kefu (~kefu@114.92.125.128) Quit (Remote host closed the connection)
[16:45] * kefu (~kefu@114.92.125.128) has joined #ceph
[16:45] * ira_away is now known as ira
[16:48] * johnavp1989 (~jpetrini@8.39.115.8) has joined #ceph
[16:48] <- *johnavp1989* To prove that you are human, please enter the result of 8+3
[16:49] * Concubidated (~cube@h4.246.129.40.static.ip.windstream.net) Quit (Quit: Leaving.)
[16:51] * vincepii (~textual@77.245.22.67) has joined #ceph
[16:51] * diver_ (~diver@95.85.8.93) has joined #ceph
[16:51] <vincepii> I guess this is the question of the day, but... what's going on with the website?
[16:54] <IcePic> VPS host sad, I think
[16:54] * peetaur2 (~peter@i4DF67CD2.pool.tripleplugandplay.com) has joined #ceph
[16:54] * diver__ (~diver@95.85.8.93) has joined #ceph
[16:55] * peetaur2 (~peter@i4DF67CD2.pool.tripleplugandplay.com) Quit ()
[16:55] * peetaur2 (~peter@i4DF67CD2.pool.tripleplugandplay.com) has joined #ceph
[16:56] * kefu (~kefu@114.92.125.128) Quit (Max SendQ exceeded)
[16:57] * kefu (~kefu@114.92.125.128) has joined #ceph
[16:58] * diver (~diver@95.85.8.93) Quit (Ping timeout: 480 seconds)
[16:59] * mykola (~Mikolaj@193.93.217.39) has joined #ceph
[17:01] * diver (~diver@95.85.8.93) has joined #ceph
[17:01] * diver_ (~diver@95.85.8.93) Quit (Ping timeout: 480 seconds)
[17:02] * wushudoin (~wushudoin@2601:646:8200:c9f0:2ab2:bdff:fe0b:a6ee) has joined #ceph
[17:02] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Remote host closed the connection)
[17:02] * analbeard (~shw@5.153.255.226) Quit (Quit: Leaving.)
[17:07] * diver__ (~diver@95.85.8.93) Quit (Ping timeout: 480 seconds)
[17:07] * ade (~abradshaw@pool-22.254.176.62.dynamic.wobline-ip.de) Quit (Ping timeout: 480 seconds)
[17:09] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[17:11] * jagardaniel1 (~oftc-webi@2001:9b0:109:103::b6) Quit (Quit: Page closed)
[17:14] * ron-slc (~Ron@173.165.129.118) Quit (Remote host closed the connection)
[17:15] * theancient (~jasonj@173-165-224-105-minnesota.hfc.comcastbusiness.net) Quit (Remote host closed the connection)
[17:15] * diver_ (~diver@216.85.162.34) has joined #ceph
[17:18] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[17:19] * diver__ (~diver@95.85.8.93) has joined #ceph
[17:20] * diver__ (~diver@95.85.8.93) Quit ()
[17:21] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[17:22] * diver (~diver@95.85.8.93) Quit (Ping timeout: 480 seconds)
[17:25] * vincepii (~textual@77.245.22.67) Quit (Quit: Textual IRC Client: www.textualapp.com)
[17:26] * diver_ (~diver@216.85.162.34) Quit (Ping timeout: 480 seconds)
[17:30] * mgolub (~Mikolaj@193.93.217.58) has joined #ceph
[17:32] * Nicho1as (~nicho1as@00022427.user.oftc.net) Quit (Quit: A man from the Far East; using WeeChat 1.5)
[17:32] * Concubidated (~cube@68.140.239.164) has joined #ceph
[17:35] * mykola (~Mikolaj@193.93.217.39) Quit (Ping timeout: 480 seconds)
[17:36] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[17:38] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) has joined #ceph
[17:38] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[17:43] <s3an2> I am using eu.ceph.com over download.ceph.com due to the issue - and noticed eu.ceph.com does not have https
[17:47] * b0e (~aledermue@213.95.25.82) Quit (Quit: Leaving.)
[17:59] * ntpttr_ (~ntpttr@192.55.54.44) has joined #ceph
[17:59] * TMM (~hp@185.5.121.201) Quit (Quit: Ex-Chat)
[18:03] * ledgr (~ledgr@88-119-196-104.static.zebra.lt) Quit (Remote host closed the connection)
[18:03] * ledgr (~ledgr@88-119-196-104.static.zebra.lt) has joined #ceph
[18:06] * ntpttr__ (~ntpttr@192.55.54.44) has joined #ceph
[18:06] * ntpttr_ (~ntpttr@192.55.54.44) Quit (Remote host closed the connection)
[18:12] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[18:17] * maybebuggy (~maybebugg@2a01:4f8:191:2350::2) Quit (Ping timeout: 480 seconds)
[18:19] * kefu (~kefu@114.92.125.128) Quit (Read error: Connection reset by peer)
[18:21] * ron-slc (~Ron@173-165-129-118-utah.hfc.comcastbusiness.net) has joined #ceph
[18:22] * branto (~branto@transit-86-181-132-209.redhat.com) Quit (Quit: ZNC 1.6.3 - http://znc.in)
[18:22] * sudocat (~dibarra@192.185.1.20) has joined #ceph
[18:23] * Unai (~Adium@2604:5500:1b:5e2:2093:1595:c080:4807) Quit (Quit: Leaving.)
[18:23] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) has joined #ceph
[18:26] * ledgr (~ledgr@88-119-196-104.static.zebra.lt) Quit (Remote host closed the connection)
[18:26] * ledgr (~ledgr@88-119-196-104.static.zebra.lt) has joined #ceph
[18:27] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) has joined #ceph
[18:28] * kefu (~kefu@114.92.125.128) has joined #ceph
[18:29] * garphy is now known as garphy`aw
[18:30] * doppelgrau (~doppelgra@132.252.235.172) Quit (Quit: Leaving.)
[18:34] * ledgr (~ledgr@88-119-196-104.static.zebra.lt) Quit (Ping timeout: 480 seconds)
[18:38] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) Quit (Ping timeout: 480 seconds)
[18:38] * blizzow (~jburns@50-243-148-102-static.hfc.comcastbusiness.net) has joined #ceph
[18:38] * kefu (~kefu@114.92.125.128) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[18:39] * dgurtner_ (~dgurtner@94.126.212.170) Quit (Ping timeout: 480 seconds)
[18:42] * derjohn_mob (~aj@46.189.28.54) Quit (Ping timeout: 480 seconds)
[18:47] * hoonetorg (~hoonetorg@fh.fh-joanneum.at) has joined #ceph
[18:48] * hbogert1 (~Adium@95.170.93.16) Quit (Ping timeout: 480 seconds)
[18:49] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[18:50] * shaunm (~shaunm@ms-208-102-105-216.gsm.cbwireless.com) has joined #ceph
[18:50] * yankcrime (~yankcrime@185.43.216.241) Quit (Ping timeout: 480 seconds)
[18:51] <blizzow> I had this really weird thing happen over the last 18 hours. Random OSD services die with a stacktrace. I'm wondering if there is some corrupt block being replicated across my cluster that's causing this. It's happened every 3-5 hours. Different drives and drive types, different OSD nodes with different specs... The network switch seems fine. Anyone have an idea? Here's a sample from the logs: http://pastebin.ca/3727584
[18:54] * wiebalck_ (~wiebalck@AAnnecy-653-1-194-38.w86-209.abo.wanadoo.fr) has joined #ceph
[18:55] <s3an2> blizzow, anything in dmesg
[18:59] * ntpttr__ (~ntpttr@192.55.54.44) Quit (Remote host closed the connection)
[18:59] <Unai> Hey guys???. is it possible to say that a PG is completely lost and unrecoverable, so don't bother trying and error out instead of timing out?
[19:00] <Unai> the OSDs that were part of it died simultaneously but the PG is still considered active+stale
[19:03] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) Quit (Quit: Leaving.)
[19:06] * DanFoster (~Daniel@2a00:1ee0:3:1337:9:fb59:d2eb:719e) Quit (Quit: Leaving)
[19:07] * xarses_ (~xarses@64.124.158.3) has joined #ceph
[19:07] <blizzow> s3an2: On one host, I saw an AIO threading error at one point. Another threw an I/O error on the drive and wouldn't recognize it. Almost like it overloaded the drive bus?
[19:10] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) has joined #ceph
[19:12] * peetaur (~peter@p200300E10BC67700667002FFFE2E10FC.dip0.t-ipconnect.de) has joined #ceph
[19:15] * TomasCZ (~TomasCZ@yes.tenlab.net) has joined #ceph
[19:18] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) has joined #ceph
[19:21] * wiebalck_ (~wiebalck@AAnnecy-653-1-194-38.w86-209.abo.wanadoo.fr) Quit (Quit: wiebalck_)
[19:22] * t4nk907 (~oftc-webi@HSI-KBW-46-223-128-43.hsi.kabel-badenwuerttemberg.de) has joined #ceph
[19:22] * Kidlvr (~tokie@104.200.154.44) has joined #ceph
[19:23] * dgurtner (~dgurtner@178.197.225.1) has joined #ceph
[19:23] * ntpttr_ (~ntpttr@134.134.139.82) has joined #ceph
[19:28] * bitserker (~toni@2.152.12.64.dyn.user.ono.com) Quit (Quit: Leaving.)
[19:30] * nicko (~nicko@173-164-42-233-colorado.hfc.comcastbusiness.net) has joined #ceph
[19:30] * nicko (~nicko@173-164-42-233-colorado.hfc.comcastbusiness.net) Quit ()
[19:30] * ntpttr_ (~ntpttr@134.134.139.82) Quit (Remote host closed the connection)
[19:30] * KindOne (kindone@0001a7db.user.oftc.net) has joined #ceph
[19:35] * mattt (~mattt@lnx1.defunct.ca) has joined #ceph
[19:36] <mattt> where can i get updates on the ceph.com outage ?
[19:37] <peetaur> mattt: I think it's http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/
[19:38] <peetaur> and are you just looking for docs, or what? You can get the docs from the git repo too
[19:39] <mattt> peetaur: i just noticed that docs, tracker, and downloads were all unavailable
[19:39] <aarontc> argh, can't search bug tracker. was hoping to find out if anyone else is getting crashes from cephfs-fuse when using '--client_mountpoint'
[19:39] * vimal (~vikumar@121.244.87.116) Quit (Quit: Leaving)
[19:41] <mattt> peetaur: eek, that doesn't sound good
[19:41] * ntpttr_ (~ntpttr@192.55.54.42) has joined #ceph
[19:42] * ntpttr_ (~ntpttr@192.55.54.42) Quit ()
[19:48] * garphy`aw is now known as garphy
[19:49] * TMM (~hp@dhcp-077-248-009-229.chello.nl) has joined #ceph
[19:52] * xinli (~charleyst@32.97.110.53) Quit (Ping timeout: 480 seconds)
[19:52] * Kidlvr (~tokie@104.200.154.44) Quit ()
[19:52] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[19:54] <Unai> anyone about how to get rid of stale PGs? I need to really mark them as down and unrecoverable.
[19:54] * evelu (~erwan@37.160.193.215) Quit (Read error: Connection reset by peer)
[20:01] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) has joined #ceph
[20:01] * keeperandy (~textual@50-245-231-209-static.hfc.comcastbusiness.net) has joined #ceph
[20:01] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[20:04] * newbie (~kvirc@host217-114-156-249.pppoe.mark-itt.net) has joined #ceph
[20:06] <wwalker> is docs.ceph.com down for anyone else?
[20:07] <peetaur> yes, see link above
[20:11] * evelu (~erwan@2a01:e34:eecb:7400:4eeb:42ff:fedc:8ac) has joined #ceph
[20:15] * dgurtner (~dgurtner@178.197.225.1) Quit (Ping timeout: 480 seconds)
[20:20] * wiebalck_ (~wiebalck@AAnnecy-653-1-194-38.w86-209.abo.wanadoo.fr) has joined #ceph
[20:28] * rotbeard (~redbeard@185.32.80.238) Quit (Quit: Leaving)
[20:28] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:a095:7e95:6611:bc7) Quit (Ping timeout: 480 seconds)
[20:29] * rendar (~I@host133-71-dynamic.171-212-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[20:35] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) Quit (Quit: Leaving.)
[20:35] * Concubidated (~cube@68.140.239.164) Quit (Remote host closed the connection)
[20:35] * Concubidated (~cube@68.140.239.164) has joined #ceph
[20:36] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) has joined #ceph
[20:36] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[20:37] * Concubidated (~cube@68.140.239.164) Quit (Remote host closed the connection)
[20:37] <s3an2> Yea, I you can still search docs on git if you really need them
[20:37] * Concubidated (~cube@68.140.239.164) has joined #ceph
[20:38] * [arx] (~arx@macha.mac-anu.org) has joined #ceph
[20:38] * hoonetorg (~hoonetorg@fh.fh-joanneum.at) Quit (Ping timeout: 480 seconds)
[20:40] <s3an2> Unai, have a look at the docs here about marking objects and osd's as lost https://goo.gl/ZUTj8n - its a google webcache link as the source site is offline
[20:43] <s3an2> blizzow, I would look at the hardware in detail as it sounds like it could be causing the asserts.
[20:43] * morourke (~Mike@2601:205:4001:561b:b0da:ab3b:1132:3d98) has joined #ceph
[20:43] * evelu (~erwan@2a01:e34:eecb:7400:4eeb:42ff:fedc:8ac) Quit (Ping timeout: 480 seconds)
[20:47] * mhack (~mhack@nat-pool-bos-u.redhat.com) has joined #ceph
[20:49] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[20:50] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) Quit (Quit: Leaving.)
[20:50] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) has joined #ceph
[20:50] * xinli (~charleyst@32.97.110.53) has joined #ceph
[20:54] * rendar (~I@host133-71-dynamic.171-212-r.retail.telecomitalia.it) has joined #ceph
[20:58] * evelu (~erwan@37.160.193.215) has joined #ceph
[21:01] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) Quit (Quit: Leaving.)
[21:03] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) has joined #ceph
[21:04] <blizzow> s3an2: I would have agreed, but it's just so strange that ceph would stop communicating four times in half a day with 3 different drive types, connected to three different motherboards, two drives with SAS RAID controllers, and two with SATA.
[21:05] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) Quit ()
[21:05] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) Quit (Remote host closed the connection)
[21:11] * jermudgeon (~jermudgeo@31.207.56.59) has joined #ceph
[21:13] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) Quit (Quit: Leaving.)
[21:16] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) has joined #ceph
[21:16] * brad_mssw (~brad@66.129.88.50) Quit (Quit: Leaving)
[21:17] * rwheeler (~rwheeler@pool-108-7-196-31.bstnma.fios.verizon.net) Quit (Quit: Leaving)
[21:21] * mhack (~mhack@nat-pool-bos-u.redhat.com) Quit (Quit: I'm outta here!)
[21:21] * mhack (~mhack@nat-pool-bos-u.redhat.com) has joined #ceph
[21:21] * mhack (~mhack@nat-pool-bos-u.redhat.com) Quit (Remote host closed the connection)
[21:22] * newdave_ (~newdave@36-209-181-180.cpe.skymesh.net.au) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[21:23] * newdave (~newdave@36-209-181-180.cpe.skymesh.net.au) has joined #ceph
[21:25] * mhack (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[21:27] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) Quit (Quit: Leaving.)
[21:32] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) has joined #ceph
[21:34] * jermudgeon (~jermudgeo@31.207.56.59) Quit (Read error: Connection reset by peer)
[21:34] * jermudgeon (~jermudgeo@31.207.56.59) has joined #ceph
[21:37] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[21:37] * mhack (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[21:41] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[21:46] * webertrlz (~oftc-webi@191.185.28.83) has left #ceph
[21:47] * salwasser (~Adium@a72-246-0-10.deploy.akamaitechnologies.com) Quit (Ping timeout: 480 seconds)
[21:48] * mgolub (~Mikolaj@193.93.217.58) Quit (Quit: away)
[21:49] * dneary (~dneary@nat-pool-bos-u.redhat.com) Quit (Quit: Ex-Chat)
[21:50] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[21:53] * jermudgeon (~jermudgeo@31.207.56.59) Quit (Ping timeout: 480 seconds)
[22:01] * etamponi (~etamponi@net-93-71-251-206.cust.vodafonedsl.it) Quit ()
[22:06] * InIMoeK (~InIMoeK@105-183-045-062.dynamic.caiway.nl) Quit (Read error: Connection reset by peer)
[22:07] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) Quit (Quit: Leaving.)
[22:08] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) has joined #ceph
[22:16] * kalmisto (~PappI@45.32.239.246) has joined #ceph
[22:22] <shaon> is ceph's website down? as well as the download page?
[22:22] <shaon> or everything has been moved to a different location?
[22:22] <lurbs> Download is for me.
[22:22] <gregsfortytwo> it's dead, Jim
[22:22] <lurbs> Yep, and the main site.
[22:23] <gregsfortytwo> the mirrors might still work, not sure how the DNS works for those
[22:23] <lurbs> http://eu.ceph.com/ is still working.
[22:23] <bstillwell> shaon: http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/
[22:23] * shaon clicks
[22:23] <lurbs> 'currently having issues with its storage system'
[22:23] <lurbs> Ironic if that's Ceph.
[22:24] <lurbs> Heh, it seems to be.
[22:24] <bstillwell> Looks like Ceph got messed up from a major networking issue
[22:24] <gregsfortytwo> it is :( but note the network outage, yeah
[22:24] <shaon> thanks folks
[22:25] <bstillwell> I've had my fair share of Ceph issues that ended up being network issues...
[22:25] <shaon> will probably wait until it's sorted out
[22:25] <shaon> bstillwell: you kept blaming your networking skills? :)
[22:25] <bstillwell> Dead hardware I can live with, but misconfigured or 'sick' hardware is a pain.
[22:26] * georgem (~Adium@206.108.127.16) Quit (Ping timeout: 480 seconds)
[22:26] <bstillwell> shaon: Heh, we have network guys for that. :)
[22:26] <shaon> haha
[22:27] <bstillwell> Dirty fiber and half-failed disks seem to cause me the most headaches.
[22:27] <lurbs> bstillwell: https://www.backblaze.com/blog/what-smart-stats-indicate-hard-drive-failures/
[22:28] <lurbs> Backblaze's advice on how to detect disks that are about to f(l)ail.
[22:28] <bstillwell> lurbs: Yep, that's good for detection. Now I just need to write the scripts to kill the drives that are sick.
[22:28] <blizzow> F*CK, I just added an OSD to one of my nodes and my cluster was rebalancing for an hour. Then suddenly Input/output error for all OSDs on that node!
[22:29] <bstillwell> blizzow: That sucks, I just head a controller corrupt 4 xfs filesystems on a node yesterday...
[22:30] <blizzow> bstillwell: In the last 24 hours, I've had 5 OSD crashes on different nodes. Different drive types, different controllers, different motherboards. I cannot for the life of me figure out WTF is going on.
[22:30] <gregsfortytwo> dirty power? somebody yelling at your servers? ;)
[22:30] <bstillwell> blizzow: bad power?
[22:31] <T1> blizzow: faulty network?
[22:31] <bstillwell> Although I have seen servers crash because a picture was taken of them with a flash...
[22:32] <bstillwell> That was because the servers had hot-swap PCI slots and the flash made them think a card was added/removed.
[22:32] * puffy (~puffy@216.207.42.140) has joined #ceph
[22:32] <blizzow> The datacenter has really nice people who would never yell at my servers. Only one of the servers does not have redunant power (off different feeds). The network is the only thing I can think of, but even that isn't causing issues for other things :/
[22:32] * oliveiradan (~doliveira@137.65.133.10) Quit (Remote host closed the connection)
[22:36] <peetaur> shouldn't the worst network in the world just cause blocked requests, not damage?
[22:36] * morourke (~Mike@2601:205:4001:561b:b0da:ab3b:1132:3d98) Quit (Quit: Leaving)
[22:37] <T1> people here have seen all sorts of strange stuff happen from faulty network - including, but not limited to OSDs dying or killing themselves
[22:37] <bstillwell> peetaur: Flakey networking cause the OSDs to flap which causes a lot of peering processes.
[22:38] <T1> and you can replace faulty network with bad firewall settings in that statement too
[22:38] <T1> and half-connected TCP sockets etc etc etc
[22:38] <T1> crazy high load
[22:40] * oliveiradan (~doliveira@137.65.133.10) has joined #ceph
[22:41] * wiebalck_ (~wiebalck@AAnnecy-653-1-194-38.w86-209.abo.wanadoo.fr) Quit (Quit: wiebalck_)
[22:43] * puffy (~puffy@216.207.42.140) Quit (Quit: Leaving.)
[22:46] * kalmisto (~PappI@45.32.239.246) Quit ()
[22:48] * KindOne_ (kindone@h197.160.186.173.dynamic.ip.windstream.net) has joined #ceph
[22:49] * Unai (~Adium@50-115-70-150.static-ip.telepacific.net) has joined #ceph
[22:50] * KindOne_ (kindone@h197.160.186.173.dynamic.ip.windstream.net) Quit ()
[22:52] * bniver (~bniver@nat-pool-bos-u.redhat.com) Quit (Remote host closed the connection)
[22:53] * ledgr (~ledgr@88-222-11-185.meganet.lt) has joined #ceph
[22:54] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[22:55] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) Quit (Quit: Leaving.)
[22:56] * vbellur (~vijay@nat-pool-bos-t.redhat.com) Quit (Quit: Leaving.)
[22:57] * davidzlap (~Adium@2605:e000:1313:8003:b91f:8dd4:bf57:96a6) has joined #ceph
[23:00] * newbie (~kvirc@host217-114-156-249.pppoe.mark-itt.net) Quit (Ping timeout: 480 seconds)
[23:01] <herrsergio> hi, is http://docs.ceph.com/ down ?
[23:01] <bstillwell> herrsergio: http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/
[23:02] * ledgr (~ledgr@88-222-11-185.meganet.lt) Quit (Ping timeout: 480 seconds)
[23:08] * t4nk907 (~oftc-webi@HSI-KBW-46-223-128-43.hsi.kabel-badenwuerttemberg.de) Quit (Quit: Page closed)
[23:10] <herrsergio> bstillwell: lol, so Ceph was the reason that the Ceph site is down
[23:11] <bstillwell> herrsergio: Looks like a network problem first which caused a Ceph problem
[23:12] * hbogert (~Adium@ip54541f88.adsl-surfen.hetnet.nl) has joined #ceph
[23:28] * jarrpa (~jarrpa@63.225.131.166) Quit (Ping timeout: 480 seconds)
[23:30] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[23:31] * peetaur (~peter@p200300E10BC67700667002FFFE2E10FC.dip0.t-ipconnect.de) Quit (Quit: Konversation terminated!)
[23:33] * evelu (~erwan@37.160.193.215) Quit (Ping timeout: 480 seconds)
[23:39] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) Quit (Ping timeout: 480 seconds)
[23:40] * atod (~atod@cpe-74-73-129-35.nyc.res.rr.com) has joined #ceph
[23:41] * johnavp1989 (~jpetrini@8.39.115.8) Quit (Ping timeout: 480 seconds)
[23:41] * evelu (~erwan@2a01:e34:eecb:7400:4eeb:42ff:fedc:8ac) has joined #ceph
[23:48] * hbogert (~Adium@ip54541f88.adsl-surfen.hetnet.nl) Quit (Quit: Leaving.)
[23:53] * pdhange (~pdhange@210.185.111.235) has joined #ceph
[23:55] <blizzow> Is there any way to make ceph play nice during a rebalance?
[23:55] <blizzow> Holy piss, I added an OSD and my entire infrastructure has gone haywire.
[23:56] <ben1> it plays pretty nice as long as you have enough disk
[23:56] <diq> blizzow, turn down the number of threads?
[23:56] <diq> I would say as long as you have enough CPU and network
[23:57] <diq> but I guess that implies enough disk
[23:57] <diq> blizzow, how many recovery threads are you using now?
[23:58] * xinli (~charleyst@32.97.110.53) Quit (Ping timeout: 480 seconds)
[23:58] <ben1> well the more disks you have the less each one gets hit
[23:58] * mhack (~mhack@nat-pool-bos-t.redhat.com) Quit (Remote host closed the connection)
[23:58] <ben1> network and cpu degredation should be smoother.
[23:58] <blizzow> diq: at one point I injected a max recovery threads of 1 into the cluster..
[23:58] * rendar (~I@host133-71-dynamic.171-212-r.retail.telecomitalia.it) Quit (Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!)
[23:59] <ben1> how many osds do you have bliz?
[23:59] <diq> can't go much lower than that ;)
[23:59] <blizzow> I have 50 OSDs.
[23:59] <ben1> hmm that should be plenty
[23:59] <ben1> maybe diq is right? :)
[23:59] <ben1> how much network do you have?

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.