#ceph IRC Log

Index

IRC Log for 2016-09-19

Timestamps are in GMT/BST.

[0:07] <doppelgrau> cetex: the sum of both AFAIK
[0:12] <cetex> ah, right
[0:12] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) Quit (Read error: Connection reset by peer)
[0:15] * vbellur (~vijay@2601:18f:700:55b0:5e51:4fff:fee8:6a5c) Quit (Ping timeout: 480 seconds)
[0:16] <cetex> so both incoming and outgoing, and it's basically a race-condition in what order pg's will be replicated, whoever is first to get it's request into the queues wins?
[0:18] <cetex> would've been nice otherwise to be able to tell nodes how aggressive they should be in pulling vs pushing data to other nodes. :)
[0:18] <cetex> since we're migrating to new nodes we want to fill those new nodes up first, and rather not do any data shuffling to the old nodes (which we'll wipe soon enough) at all.
[0:19] <cetex> .. As long as nr of copies of data is met.
[0:27] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[0:38] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) Quit (Quit: doppelgrau)
[0:40] * ron-slc_ (~Ron@173-165-129-118-utah.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[0:41] * ron-slc_ (~Ron@173-165-129-118-utah.hfc.comcastbusiness.net) has joined #ceph
[0:52] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[0:58] * Kingrat (~shiny@2605:6000:1526:4063:ecdf:a098:2871:dc2c) Quit (Ping timeout: 480 seconds)
[1:02] * vbellur (~vijay@2601:18f:700:55b0:5e51:4fff:fee8:6a5c) has joined #ceph
[1:07] * Kingrat (~shiny@2605:6000:1526:4063:e12d:b41d:2004:d397) has joined #ceph
[1:15] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[1:23] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[1:24] * [0x4A6F]_ (~ident@p4FC26831.dip0.t-ipconnect.de) has joined #ceph
[1:25] * ron-slc_ (~Ron@173-165-129-118-utah.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[1:27] * ron-slc_ (~Ron@173-165-129-118-utah.hfc.comcastbusiness.net) has joined #ceph
[1:27] * [0x4A6F] (~ident@0x4a6f.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:27] * [0x4A6F]_ is now known as [0x4A6F]
[1:35] * xinze (~xinzechi@211.97.126.103) has joined #ceph
[1:36] * xinze (~xinzechi@211.97.126.103) has left #ceph
[1:39] * Kurimus1 (~lobstar@tsn109-201-152-227.dyn.nltelcom.net) has joined #ceph
[1:43] * oms101 (~oms101@p20030057EA500100C6D987FFFE4339A1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[1:50] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[1:52] * oms101 (~oms101@p20030057EA018300C6D987FFFE4339A1.dip0.t-ipconnect.de) has joined #ceph
[2:04] * Kurimus1 (~lobstar@tsn109-201-152-227.dyn.nltelcom.net) Quit (Ping timeout: 480 seconds)
[2:07] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[2:16] * CustosLimen (~CustosLim@2001:41d0:1:ff97::1) has joined #ceph
[2:21] * salwasser (~Adium@c-76-118-229-231.hsd1.ma.comcast.net) has joined #ceph
[2:26] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[2:30] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[2:31] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) has joined #ceph
[2:32] * wkennington (~wkenningt@c-71-204-170-241.hsd1.ca.comcast.net) has joined #ceph
[2:35] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) Quit (Read error: Connection reset by peer)
[2:46] * srk (~Siva@2605:6000:ed04:ce00:6459:75df:ed98:5f5d) has joined #ceph
[2:49] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[3:00] * Doodlepieguy (~vegas3@185.3.135.114) has joined #ceph
[3:00] * srk (~Siva@2605:6000:ed04:ce00:6459:75df:ed98:5f5d) Quit (Ping timeout: 480 seconds)
[3:08] * kuku (~kuku@119.93.91.136) has joined #ceph
[3:11] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[3:12] * jfaj (~jan@p4FC258C9.dip0.t-ipconnect.de) has joined #ceph
[3:16] * Pulp (~Pulp@63-221-50-195.dyn.estpak.ee) has joined #ceph
[3:18] * valeech (~valeech@pool-96-247-203-33.clppva.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[3:19] * jfaj__ (~jan@p4FC250C7.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[3:19] * valeech (~valeech@pool-96-247-203-33.clppva.fios.verizon.net) has joined #ceph
[3:21] * derjohn_mobi (~aj@x590e4b13.dyn.telefonica.de) has joined #ceph
[3:28] * derjohn_mob (~aj@x590c56c2.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[3:28] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[3:30] * Doodlepieguy (~vegas3@185.3.135.114) Quit ()
[3:31] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[3:41] * sebastian-w_ (~quassel@212.218.8.138) Quit (Remote host closed the connection)
[3:41] * sebastian-w (~quassel@212.218.8.138) has joined #ceph
[3:49] * kefu (~kefu@114.92.125.128) has joined #ceph
[3:54] * salwasser (~Adium@c-76-118-229-231.hsd1.ma.comcast.net) Quit (Quit: Leaving.)
[4:01] * kefu is now known as kefu|afk
[4:05] * kefu|afk is now known as kefu
[4:06] <ronrib> I think I'm missing something, what's the difference between mellanox sx1036 vs sx1710? They seem to be different architectures but they should do the same job right?
[4:07] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[4:10] * srk (~Siva@2605:6000:ed04:ce00:69e1:58ec:1a8:1d05) has joined #ceph
[4:19] * datagutt (~mLegion@46.166.137.245) has joined #ceph
[4:23] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[4:31] * srk_ (~Siva@2605:6000:ed04:ce00:9436:2d2e:54e7:14a3) has joined #ceph
[4:36] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[4:37] * srk (~Siva@2605:6000:ed04:ce00:69e1:58ec:1a8:1d05) Quit (Ping timeout: 480 seconds)
[4:49] * datagutt (~mLegion@46.166.137.245) Quit ()
[4:51] * ph470m (~ph470m@74.50.197.36) has joined #ceph
[4:51] * ph470m (~ph470m@74.50.197.36) has left #ceph
[4:56] * kefu (~kefu@114.92.125.128) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[4:57] * kefu (~kefu@114.92.125.128) has joined #ceph
[5:07] * EinstCra_ (~EinstCraz@58.247.119.250) has joined #ceph
[5:07] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Read error: Connection reset by peer)
[5:09] * flisky (~Thunderbi@106.38.61.185) has joined #ceph
[5:12] * Borf (~Doodlepie@108.61.123.66) has joined #ceph
[5:15] * srk_ (~Siva@2605:6000:ed04:ce00:9436:2d2e:54e7:14a3) Quit (Ping timeout: 480 seconds)
[5:18] * Vacuum__ (~Vacuum@88.130.214.192) has joined #ceph
[5:25] * Vacuum_ (~Vacuum@i59F790E4.versanet.de) Quit (Ping timeout: 480 seconds)
[5:27] * srk_ (~Siva@cpe-70-113-23-93.austin.res.rr.com) has joined #ceph
[5:32] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[5:36] * malevolent (~quassel@192.146.172.118) Quit (Read error: Connection reset by peer)
[5:36] * srk_ (~Siva@cpe-70-113-23-93.austin.res.rr.com) Quit (Read error: Connection reset by peer)
[5:37] * malevolent (~quassel@192.146.172.118) has joined #ceph
[5:37] * flisky (~Thunderbi@106.38.61.185) Quit (Quit: flisky)
[5:38] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) has joined #ceph
[5:42] * Borf (~Doodlepie@5AEAABQ0Q.tor-irc.dnsbl.oftc.net) Quit ()
[5:55] * SweetGirl (~Solvius@exit0.radia.tor-relays.net) has joined #ceph
[5:56] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) Quit (Ping timeout: 480 seconds)
[6:09] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[6:11] * walcubi (~walcubi@p5795B5B3.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[6:11] * walcubi (~walcubi@p5795B501.dip0.t-ipconnect.de) has joined #ceph
[6:24] * nhm (~nhm@c-50-171-139-246.hsd1.mn.comcast.net) Quit (Ping timeout: 480 seconds)
[6:25] * SweetGirl (~Solvius@5AEAABQ1G.tor-irc.dnsbl.oftc.net) Quit ()
[6:34] * Teddybareman (~utugi____@213.61.149.100) has joined #ceph
[6:38] * derjohn_mobi (~aj@x590e4b13.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[6:42] * TomasCZ (~TomasCZ@yes.tenlab.net) Quit (Quit: Leaving)
[6:50] * t4nk417 (~oftc-webi@117.247.186.15) has joined #ceph
[6:52] <t4nk417> Hi I would like to move existing OSD journals on production ceph to SSD disks. Each OSD is of 1TB. I have 4 OSDs. Is there any dwontime associated with this or can I do it on the fly.
[6:53] * rdas (~rdas@121.244.87.116) has joined #ceph
[7:01] <t4nk417> How much time will it take to recreate journal
[7:03] * tsg (~tgohad@192.55.54.40) Quit (Ping timeout: 480 seconds)
[7:04] * Teddybareman (~utugi____@26XAAB05L.tor-irc.dnsbl.oftc.net) Quit ()
[7:08] <lurbs> If you take down the OSDs one at a time there's no downtime to swap over to a journal on an SSD, just slightly degraded redundancy during that period.
[7:08] <sep> t4nk417, you will need to stop the osd in question
[7:09] <lurbs> The actual process of stopping the OSD, flushing the journal, creating a new one, and starting it should only take a few minutes.
[7:09] <sep> but just in case set noout so you do not start any recovery unneeded.
[7:09] <lurbs> I'd recommend trialling it on a test cluster first, to ensure you get the process right.
[7:10] <t4nk417> Can I just delete the old journal /var/lib/ceph/osd/<osd-id>/journal
[7:10] <t4nk417> and make a symlink to ssd
[7:10] <sep> t4nk417, yes. just make sure you flush the journal first
[7:10] <t4nk417> ln -s /var/lib/ceph/osd/<osd-id>/journal /dev/<ssd-partition-for-your-journal>
[7:10] <t4nk417> THanks so much
[7:10] <lurbs> You'd do that in between the flush and the mkjournal, yes.
[7:12] <sep> http://paste.debian.net/828978/ ; this is the script that i used for each of mine
[7:14] <sep> t4nk417, and i would reccomend to use a uuid for the journal disk and not /dev/sdXn since those might change during reboots
[7:15] <t4nk417> Oaky
[7:16] <t4nk417> okay
[7:16] * lixiaoy1 (~lixiaoy1@192.102.204.38) has joined #ceph
[7:16] <t4nk417> Is there any recomended parition size for SSD journals
[7:17] <t4nk417> OSD is of 1TB
[7:17] <sep> there is a formula for it.
[7:17] * icey (~Chris@0001bbad.user.oftc.net) Quit (Ping timeout: 480 seconds)
[7:18] <sep> http://docs.ceph.com/docs/jewel/rados/configuration/osd-config-ref/ ;; The journal size should be at least twice the product of the expected drive speed multiplied by filestore max sync interval.
[7:18] * icey (~Chris@pool-71-162-145-72.phlapa.fios.verizon.net) has joined #ceph
[7:18] <t4nk417> I was comfused with the drive speed
[7:19] * Skaag1 (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Read error: Connection reset by peer)
[7:19] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[7:20] * eth00- (~eth00@74.81.187.100) Quit (Read error: Connection reset by peer)
[7:20] * eth00 (~eth00@74.81.187.100) has joined #ceph
[7:21] <sep> imagine you have traditional spinners at 200MB/s and your sync interval is default 5 secs then 2*5*200 = 2000 MB
[7:22] <sep> but the default is 5GB, i have 15GB and i do not think it's any point to have it smaller then that, since you have plenty of place
[7:23] <t4nk417> I will check for the drive speed then
[7:24] * kefu is now known as kefu|afk
[7:24] <t4nk417> Previosly I used 15GB for journal partition
[7:25] <sep> if it's spinning rust then that's sufficient
[7:26] * jlayton (~jlayton@cpe-2606-A000-1125-405B-14D9-DFF4-8FF1-7DD8.dyn6.twc.com) Quit (Remote host closed the connection)
[7:27] * med (~medberry@71.74.177.250) Quit (Ping timeout: 480 seconds)
[7:27] * med (~medberry@71.74.177.250) has joined #ceph
[7:29] <t4nk417> So I will take down one osd at a time
[7:29] <t4nk417> THank you for your help
[7:29] <t4nk417> :)
[7:30] * jlayton (~jlayton@cpe-2606-A000-1125-405B-14D9-DFF4-8FF1-7DD8.dyn6.twc.com) has joined #ceph
[7:34] * karnan (~karnan@125.16.34.66) has joined #ceph
[7:35] * kefu|afk (~kefu@114.92.125.128) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[7:36] * kefu (~kefu@114.92.125.128) has joined #ceph
[7:53] * derjohn_mob (~aj@tmo-108-163.customers.d1-online.com) has joined #ceph
[7:54] * TheSov3 (~TheSov@108-75-213-57.lightspeed.cicril.sbcglobal.net) has joined #ceph
[7:58] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) has joined #ceph
[7:58] * t4nk417 (~oftc-webi@117.247.186.15) Quit (Quit: Page closed)
[8:00] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) has joined #ceph
[8:01] * TheSov2 (~TheSov@108-75-213-57.lightspeed.cicril.sbcglobal.net) Quit (Ping timeout: 480 seconds)
[8:03] * kefu (~kefu@114.92.125.128) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[8:04] * nathani (~nathani@2607:f2f8:ac88::) has joined #ceph
[8:06] * karnan (~karnan@125.16.34.66) Quit (Quit: Leaving)
[8:06] * karnan (~karnan@125.16.34.66) has joined #ceph
[8:06] * nathani (~nathani@2607:f2f8:ac88::) Quit ()
[8:06] * nathani (~nathani@2607:f2f8:ac88::) has joined #ceph
[8:07] * nathani (~nathani@2607:f2f8:ac88::) Quit ()
[8:08] * nathani (~nathani@2607:f2f8:ac88::) has joined #ceph
[8:17] * lmb (~Lars@2a02:8109:8100:1d2c:2ad2:44ff:fedf:3318) has joined #ceph
[8:29] * EinstCra_ (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[8:31] * EinstCrazy (~EinstCraz@58.247.117.134) has joined #ceph
[8:32] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Quit: Leaving.)
[8:34] * derjohn_mob (~aj@tmo-108-163.customers.d1-online.com) Quit (Ping timeout: 480 seconds)
[8:37] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[8:37] * Diablothein (~Szernex@exit1.radia.tor-relays.net) has joined #ceph
[8:39] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit ()
[8:40] * LiamMon (~liam.monc@163.172.181.66) Quit (Remote host closed the connection)
[8:44] * Kurt (~Adium@2001:628:1:5:e460:44e2:5a4f:a9db) has joined #ceph
[8:53] * kefu (~kefu@114.92.125.128) has joined #ceph
[8:55] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[8:59] * karnan (~karnan@125.16.34.66) Quit (Remote host closed the connection)
[9:01] * karnan (~karnan@125.16.34.66) has joined #ceph
[9:02] * schegi (~schegi@81.169.147.212) Quit (Quit: leaving)
[9:03] * schegi (~schegi@81.169.147.212) has joined #ceph
[9:03] * schegi (~schegi@81.169.147.212) Quit ()
[9:05] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Quit: Leaving.)
[9:05] * EinstCrazy (~EinstCraz@58.247.117.134) Quit (Remote host closed the connection)
[9:07] * Diablothein (~Szernex@5AEAABQ4H.tor-irc.dnsbl.oftc.net) Quit ()
[9:07] * clusterfudge (~Bobby@185.65.134.77) has joined #ceph
[9:09] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[9:11] * sleinen (~Adium@2001:620:1000:4:a65e:60ff:fedb:f305) has joined #ceph
[9:15] * Guest751 is now known as zigo
[9:15] * kefu (~kefu@114.92.125.128) Quit (Max SendQ exceeded)
[9:15] * clusterfudge (~Bobby@635AAAOSQ.tor-irc.dnsbl.oftc.net) Quit (Ping timeout: 480 seconds)
[9:16] * kefu (~kefu@114.92.125.128) has joined #ceph
[9:16] * derjohn_mob (~aj@46.189.28.70) has joined #ceph
[9:24] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[9:36] * stefan_ (~stefan@ip-185-87-117-140.fiber.nl) Quit (Quit: stefan_)
[9:38] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) Quit (Quit: doppelgrau)
[9:39] * TheSov3 (~TheSov@108-75-213-57.lightspeed.cicril.sbcglobal.net) Quit (Quit: Leaving)
[9:41] * sleinen (~Adium@2001:620:1000:4:a65e:60ff:fedb:f305) Quit (Quit: Leaving.)
[9:44] * fsimonce (~simon@host98-71-dynamic.1-87-r.retail.telecomitalia.it) has joined #ceph
[9:45] <walcubi> Hmm, I'm getting an assert in PG::_peek_map_epoch when replaying the journal after a crashed OSD.
[9:46] <walcubi> The omap directory was corrupt so I moved away all ldb files to let it recreate them.
[9:47] <walcubi> Should the journal be flushed independently before restarting?
[9:47] * ggarg (~ggarg@host-82-135-29-34.customer.m-online.net) has joined #ceph
[9:48] * kefu (~kefu@114.92.125.128) Quit (Max SendQ exceeded)
[9:48] <walcubi> Nope, that's no good
[9:49] <Be-El> if just one osd has crashed, i would reformat it and build it from scratch
[9:49] * kefu (~kefu@114.92.125.128) has joined #ceph
[9:49] * ZombieTree (~BlS@exit0.liskov.tor-relays.net) has joined #ceph
[9:51] * nils_ (~nils_@doomstreet.collins.kg) has joined #ceph
[9:55] <walcubi> And the data?
[9:55] * tsg (~tgohad@134.134.139.77) has joined #ceph
[9:55] <Be-El> that's what the replication on ceph is for
[9:56] * stefan_ (~stefan@89.207.24.152) has joined #ceph
[9:56] <Kvisle_> well, there's the possibility that walcubi may have put a size or min_size of 1 somewhere
[9:57] * Kvisle_ is now known as Kvisle
[9:57] <IcePic> Kvisle_: if so, then data probably just is lost, same as if you have no backups on a normal disk and indexes/dir-structure gets whacked.
[9:58] <walcubi> Well, replication is precisely what caused this mess. :-P
[10:00] <walcubi> That and the filestore backend decided to grow 1TB in size within a few hours, knocking out many instances with out of space errors.
[10:00] * wkennington (~wkenningt@c-71-204-170-241.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[10:01] * DanFoster (~Daniel@2a00:1ee0:3:1337:21cd:fc9f:b37d:e8e0) has joined #ceph
[10:01] <walcubi> Be-El, I could just mv current current.lifesupport
[10:01] <walcubi> start up ceph
[10:02] <walcubi> Then do an rsync-like copy back?
[10:03] * sleinen1 (~Adium@eduroam-mapped-235.ethz.ch) has joined #ceph
[10:03] <walcubi> A la: rados_write_full(key, data); unlink(path)
[10:03] <Be-El> walcubi: what exactly do you want to copy?
[10:03] <walcubi> Data that is still physically on disk
[10:03] <walcubi> But not in ceph
[10:04] <walcubi> I think this should work
[10:04] <walcubi> One object == one file.
[10:04] * branto (~branto@178.253.167.12) has joined #ceph
[10:04] * nardial (~ls@p5DC0715A.dip0.t-ipconnect.de) has joined #ceph
[10:05] <Be-El> walcubi: osd do scan the current directory upon startup and make an inventory of all objects stored
[10:05] <walcubi> But the osd asserts(value == 2)
[10:05] <Be-El> walcubi: if your leveldb is broken, you either need to repair it, or use the easy way and just reformat the osd and set it up again
[10:06] <Be-El> (given that you do not use size=1 pools....)
[10:06] <walcubi> I'm also looking for repairing tools, but it's a bit patchy
[10:06] <walcubi> Most of the documentation is in git commits.
[10:12] * schegi (~schegi@81.169.147.212) has joined #ceph
[10:14] * sudocat (~dibarra@cpe-76-173-12-203.hawaii.res.rr.com) has joined #ceph
[10:17] * LiamMon (~liam.monc@electro.moncur.eu) has joined #ceph
[10:19] * ZombieTree (~BlS@26XAAB09O.tor-irc.dnsbl.oftc.net) Quit ()
[10:22] * derjohn_mob (~aj@46.189.28.70) Quit (Read error: Connection reset by peer)
[10:22] * sudocat (~dibarra@cpe-76-173-12-203.hawaii.res.rr.com) Quit (Ping timeout: 480 seconds)
[10:22] * TMM (~hp@dhcp-077-248-009-229.chello.nl) Quit (Quit: Ex-Chat)
[10:22] * derjohn_mob (~aj@46.189.28.39) has joined #ceph
[10:23] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:405b:e9e5:f08e:901e) has joined #ceph
[10:27] * rraja (~rraja@125.16.34.66) has joined #ceph
[10:30] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[10:30] * rendar (~I@host92-69-dynamic.171-212-r.retail.telecomitalia.it) has joined #ceph
[10:35] * mattch (~mattch@w5430.see.ed.ac.uk) has joined #ceph
[10:37] * rakeshgm (~rakesh@125.16.34.66) has joined #ceph
[10:40] * kjetijor_ (kjetijor@hildring.pvv.ntnu.no) Quit (Ping timeout: 480 seconds)
[10:48] * schegi (~schegi@81.169.147.212) Quit (Quit: leaving)
[10:48] * schegi (~schegi@81.169.147.212) has joined #ceph
[10:49] * Dw_Sn (~Dw_Sn@00020a72.user.oftc.net) has joined #ceph
[10:49] * schegi (~schegi@81.169.147.212) Quit ()
[10:50] * effractur (~Erik@hlm000.nl.z4p.nl) has joined #ceph
[10:51] * schegi (~schegi@81.169.147.212) has joined #ceph
[10:57] * nardial (~ls@p5DC0715A.dip0.t-ipconnect.de) Quit (Quit: Leaving)
[10:59] * lixiaoy1 (~lixiaoy1@192.102.204.38) Quit (Remote host closed the connection)
[10:59] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[11:00] <effractur> any idea if there they are plans to release ppc64le packages?
[11:02] <walcubi> Hmm, rebuild osd won't come up
[11:05] * eth00- (~eth00@74.81.187.100) has joined #ceph
[11:06] * valeech_ (~valeech@pool-96-247-203-33.clppva.fios.verizon.net) has joined #ceph
[11:06] * tsg_ (~tgohad@192.55.54.40) has joined #ceph
[11:06] * abhishekvrshny_ (uid185733@id-185733.charlton.irccloud.com) has joined #ceph
[11:06] * scalability-junk_ (sid6422@id-6422.ealing.irccloud.com) has joined #ceph
[11:07] * icey (~Chris@0001bbad.user.oftc.net) Quit (Read error: Connection reset by peer)
[11:07] <darkfader> effractur: i just read ppc604e for a moment
[11:07] <darkfader> times...
[11:08] * rakeshgm (~rakesh@125.16.34.66) Quit (Ping timeout: 480 seconds)
[11:08] <effractur> ?
[11:09] * abhishekvrshny (uid185733@id-185733.charlton.irccloud.com) Quit (Ping timeout: 480 seconds)
[11:09] * abhishekvrshny_ is now known as abhishekvrshny
[11:10] * eth00 (~eth00@74.81.187.100) Quit (Ping timeout: 480 seconds)
[11:10] * valeech (~valeech@pool-96-247-203-33.clppva.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[11:10] * valeech_ is now known as valeech
[11:10] * scalability-junk (sid6422@id-6422.ealing.irccloud.com) Quit (Ping timeout: 480 seconds)
[11:10] * scalability-junk_ is now known as scalability-junk
[11:11] * icey (~Chris@pool-71-162-145-72.phlapa.fios.verizon.net) has joined #ceph
[11:11] * tsg (~tgohad@134.134.139.77) Quit (Ping timeout: 480 seconds)
[11:12] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:14] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[11:14] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) Quit (Quit: Leaving.)
[11:16] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) has joined #ceph
[11:18] * rakeshgm (~rakesh@125.16.34.66) has joined #ceph
[11:22] * sleinen1 (~Adium@eduroam-mapped-235.ethz.ch) Quit (Quit: Leaving.)
[11:24] * kuku (~kuku@119.93.91.136) Quit (Quit: computer sleep)
[11:26] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:26] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[11:30] * rakeshgm (~rakesh@125.16.34.66) Quit (Quit: Peace :))
[11:31] <walcubi> An osd wouldn't refuse to start because the fs is near-full, right?
[11:31] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:32] <walcubi> Or maybe I did something wrong when rebuilding it by hand...
[11:33] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[11:36] * sleinen (~Adium@macsl.switch.ch) has joined #ceph
[11:36] <walcubi> Ah, there we go. :=)
[11:36] <walcubi> Or maybe not...
[11:40] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[11:42] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:42] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[11:45] <walcubi> And there seems to be something else strange possibly happening.
[11:45] <walcubi> ceph osd df reports a different size to df
[11:46] <walcubi> It thinks it is less full that what it actually is.
[11:46] * rwheeler (~rwheeler@pool-173-48-195-215.bstnma.fios.verizon.net) has joined #ceph
[11:48] * dan__ (~Daniel@2a00:1ee0:3:1337:c92d:d224:d709:500f) has joined #ceph
[11:48] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:48] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[11:48] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:49] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[11:54] * TMM (~hp@185.5.121.201) has joined #ceph
[11:55] * DanFoster (~Daniel@2a00:1ee0:3:1337:21cd:fc9f:b37d:e8e0) Quit (Ping timeout: 480 seconds)
[11:59] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[11:59] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[12:21] * stefan_ (~stefan@89.207.24.152) Quit (Quit: stefan_)
[12:38] * karnan (~karnan@125.16.34.66) Quit (Ping timeout: 480 seconds)
[12:38] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[12:39] * Lokta (~Lokta@carbon.coe.int) has joined #ceph
[12:50] * karnan (~karnan@125.16.34.66) has joined #ceph
[12:54] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[12:55] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[12:59] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Read error: Connection reset by peer)
[12:59] * ivve (~zed@cust-gw-11.se.zetup.net) has joined #ceph
[13:04] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) Quit (Quit: Leaving)
[13:09] * T1 (~the_one@5.186.54.143) Quit (Read error: Connection reset by peer)
[13:10] * T1 (~the_one@5.186.54.143) has joined #ceph
[13:17] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[13:18] * briner (~briner@2001:620:600:1000:11c:c342:6be9:dab7) Quit (Remote host closed the connection)
[13:21] * karnan (~karnan@125.16.34.66) Quit (Ping timeout: 480 seconds)
[13:25] * tsg_ (~tgohad@192.55.54.40) Quit (Remote host closed the connection)
[13:26] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) Quit (Ping timeout: 480 seconds)
[13:30] * karnan (~karnan@125.16.34.66) has joined #ceph
[13:31] * jcsp (~jspray@82-71-16-249.dsl.in-addr.zen.co.uk) has joined #ceph
[13:32] * mhack (~mhack@24-151-36-149.dhcp.nwtn.ct.charter.com) has joined #ceph
[13:37] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[13:38] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[13:39] * nils_ (~nils_@doomstreet.collins.kg) Quit (Quit: This computer has gone to sleep)
[13:41] * rraja (~rraja@125.16.34.66) Quit (Quit: Leaving)
[13:42] * pdrakeweb (~pdrakeweb@pool-98-118-150-184.bflony.fios.verizon.net) has joined #ceph
[13:42] <ndru_> Adding a new OSD to the cluster, it's been awhile since I did it. The disk/journal seem fine, the daemon runs, it shows a total number of OSDs, but it's neither "up" nor "in". Where should I begin to investigate it further? Log entries for the OSD also seem fine.
[13:44] <BranchPredictor> ndru_: is it in the crush map?
[13:45] <ndru_> BranchPredictor: It appears different from the rest in the crush map
[13:46] <ndru_> 3 0.50000 osd.3 down 0 1.00000
[13:46] <peetaur2> so...I found a bug in 0.94.9 that is fixed in 12.2.2; shouldn't 0.94.x get a backport of that? or is it fixed by mistake and I should report it? (I can corrupt files on cephfs)
[13:47] * kefu (~kefu@114.92.125.128) Quit (Max SendQ exceeded)
[13:47] * kefu (~kefu@114.92.125.128) has joined #ceph
[13:53] * valeech (~valeech@pool-96-247-203-33.clppva.fios.verizon.net) Quit (Quit: valeech)
[13:55] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) Quit (Quit: Leaving)
[13:56] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[13:56] * karnan (~karnan@125.16.34.66) Quit (Quit: Leaving)
[13:56] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) has joined #ceph
[13:57] * dugravot61 (~dugravot6@nat-persul-plg.wifi.univ-lorraine.fr) has joined #ceph
[13:57] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) Quit ()
[13:58] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[13:59] * jcsp (~jspray@82-71-16-249.dsl.in-addr.zen.co.uk) Quit (Quit: Ex-Chat)
[13:59] * icey (~Chris@0001bbad.user.oftc.net) Quit (Remote host closed the connection)
[14:00] * icey (~Chris@pool-71-162-145-72.phlapa.fios.verizon.net) has joined #ceph
[14:01] * nils_ (~nils_@doomstreet.collins.kg) has joined #ceph
[14:02] * dugravot6 (~dugravot6@l-p-dn-in-4a.lionnois.site.univ-lorraine.fr) Quit (Ping timeout: 480 seconds)
[14:03] * porunov (~alex@109.86.184.197) has joined #ceph
[14:03] * rraja (~rraja@125.16.34.66) has joined #ceph
[14:03] * porunov (~alex@109.86.184.197) has left #ceph
[14:03] * sleinen1 (~Adium@130.59.94.132) has joined #ceph
[14:04] * dugravot61 (~dugravot6@nat-persul-plg.wifi.univ-lorraine.fr) Quit (Quit: Leaving.)
[14:06] * karnan (~karnan@125.16.34.66) has joined #ceph
[14:08] * sleinen (~Adium@macsl.switch.ch) Quit (Ping timeout: 480 seconds)
[14:08] * sleinen (~Adium@2001:620:0:82::104) has joined #ceph
[14:09] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) Quit (Ping timeout: 480 seconds)
[14:13] * sleinen1 (~Adium@130.59.94.132) Quit (Ping timeout: 480 seconds)
[14:20] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[14:24] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[14:27] * shaunm (~shaunm@ms-208-102-105-216.gsm.cbwireless.com) Quit (Ping timeout: 480 seconds)
[14:28] * tsg_ (~tgohad@192.55.54.43) has joined #ceph
[14:31] * malevolent (~quassel@192.146.172.118) Quit (Ping timeout: 480 seconds)
[14:35] * Hemanth (~hkumar_@125.16.34.66) has joined #ceph
[14:36] * nhm (~nhm@c-50-171-139-246.hsd1.mn.comcast.net) has joined #ceph
[14:36] * ChanServ sets mode +o nhm
[14:38] * Pulp (~Pulp@63-221-50-195.dyn.estpak.ee) Quit (Read error: Connection reset by peer)
[14:41] * kefu is now known as kefu|afk
[14:43] * yanzheng (~zhyan@118.116.115.254) has joined #ceph
[14:44] <singler> hey yanzheng
[14:49] * tsg_ (~tgohad@192.55.54.43) Quit (Remote host closed the connection)
[14:51] * mr_flea (~LRWerewol@213.61.149.100) has joined #ceph
[14:52] <yanzheng> hi
[14:52] <yanzheng> singler,
[14:53] <singler> let's PM? Or keep conversation in channel?
[14:54] * tsg_ (~tgohad@192.55.54.43) has joined #ceph
[14:59] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) has joined #ceph
[15:00] * kefu|afk (~kefu@114.92.125.128) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[15:02] * Hemanth (~hkumar_@125.16.34.66) Quit (Ping timeout: 480 seconds)
[15:05] * branto (~branto@178.253.167.12) Quit (Ping timeout: 480 seconds)
[15:07] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) Quit (Read error: Connection reset by peer)
[15:08] * tsg_ (~tgohad@192.55.54.43) Quit (Remote host closed the connection)
[15:09] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[15:11] * donatas (~donatas@88-119-196-104.static.zebra.lt) has joined #ceph
[15:12] <donatas> hi, what should I look first to debug ceph slowness? I'm getting only 80MB/s, that's weird..
[15:13] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) has joined #ceph
[15:15] * branto (~branto@178.253.167.12) has joined #ceph
[15:19] * georgem (~Adium@206.108.127.16) has joined #ceph
[15:20] * mr_flea (~LRWerewol@26XAAB1HA.tor-irc.dnsbl.oftc.net) Quit ()
[15:23] * srk (~Siva@cpe-70-113-23-93.austin.res.rr.com) Quit (Read error: Connection reset by peer)
[15:23] * dugravot6 (~dugravot6@l-p-dn-in-4a.lionnois.site.univ-lorraine.fr) has joined #ceph
[15:25] * Racpatel (~Racpatel@2601:87:3:31e3::77ec) has joined #ceph
[15:26] * srk (~Siva@2605:6000:ed04:ce00:68ca:1b14:20ac:9299) has joined #ceph
[15:29] * bniver (~bniver@71-9-144-29.static.oxfr.ma.charter.com) has joined #ceph
[15:37] <donatas> https://gist.github.com/ton31337/1a6801008af5aa531ef828b8097faeb1
[15:37] <donatas> is it normal? almost fifth futex() is timeouting
[15:38] * srk (~Siva@2605:6000:ed04:ce00:68ca:1b14:20ac:9299) Quit (Ping timeout: 480 seconds)
[15:39] * _303 (~LRWerewol@tor2r.ins.tor.net.eu.org) has joined #ceph
[15:45] * salwasser (~Adium@72.246.3.14) has joined #ceph
[15:46] * tsg (~tgohad@192.55.54.40) has joined #ceph
[15:49] * _303 (~LRWerewol@tor2r.ins.tor.net.eu.org) Quit (Remote host closed the connection)
[15:49] * tsg (~tgohad@192.55.54.40) Quit (Remote host closed the connection)
[15:49] * tsg (~tgohad@134.134.139.74) has joined #ceph
[15:53] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[15:54] * dgurtner (~dgurtner@84-73-130-19.dclient.hispeed.ch) has joined #ceph
[15:56] * stefan_ (~stefan@89.207.24.152) has joined #ceph
[15:58] * andreww (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[15:59] * mattbenjamin (~mbenjamin@76-206-42-50.lightspeed.livnmi.sbcglobal.net) has joined #ceph
[16:00] * shaunm (~shaunm@cpe-192-180-17-174.kya.res.rr.com) has joined #ceph
[16:01] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[16:04] * squizzi (~squizzi@107.13.237.240) has joined #ceph
[16:05] * thomnico (~thomnico@cro38-2-88-180-16-18.fbx.proxad.net) has joined #ceph
[16:06] * vata (~vata@207.96.182.162) has joined #ceph
[16:10] * thomnico (~thomnico@cro38-2-88-180-16-18.fbx.proxad.net) Quit (Remote host closed the connection)
[16:10] * thomnico (~thomnico@cro38-2-88-180-16-18.fbx.proxad.net) has joined #ceph
[16:11] * thomnico (~thomnico@cro38-2-88-180-16-18.fbx.proxad.net) Quit ()
[16:14] * thomnico (~thomnico@cro38-2-88-180-16-18.fbx.proxad.net) has joined #ceph
[16:18] * tsg (~tgohad@134.134.139.74) Quit (Ping timeout: 480 seconds)
[16:19] * andreww (~xarses@64.124.158.3) has joined #ceph
[16:19] * ircolle (~Adium@2601:285:201:633a:d0d2:c6dd:fe8c:a356) has joined #ceph
[16:21] * tsg (~tgohad@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[16:24] * neurodrone (~neurodron@162.243.191.67) has joined #ceph
[16:27] * thomnico (~thomnico@cro38-2-88-180-16-18.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[16:28] * dneary (~dneary@157.157.58.170) has joined #ceph
[16:35] * thomnico (~thomnico@2a01:e35:8b41:120:5528:ea44:6ab4:71ed) has joined #ceph
[16:41] * lpabon (~quassel@nat-pool-bos-t.redhat.com) has joined #ceph
[16:42] * dneary (~dneary@157.157.58.170) Quit (Ping timeout: 480 seconds)
[16:45] * Freeaqingme (~quassel@nl3.s.kynet.eu) Quit (Quit: No Ping reply in 180 seconds.)
[16:45] * kefu (~kefu@114.92.125.128) has joined #ceph
[16:47] * Freeaqingme (~quassel@nl3.s.kynet.eu) has joined #ceph
[16:52] * mattbenjamin (~mbenjamin@76-206-42-50.lightspeed.livnmi.sbcglobal.net) Quit (Quit: Leaving.)
[16:53] * kefu (~kefu@114.92.125.128) Quit (Max SendQ exceeded)
[16:53] * haplo37 (~haplo37@199.91.185.156) has joined #ceph
[16:54] * tsg (~tgohad@jfdmzpr05-ext.jf.intel.com) Quit (Ping timeout: 480 seconds)
[16:54] * srk (~Siva@32.97.110.56) has joined #ceph
[16:54] * kefu (~kefu@114.92.125.128) has joined #ceph
[16:54] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[16:55] * TMM (~hp@185.5.121.201) Quit (Quit: Ex-Chat)
[16:56] * joshd (~jdurgin@2602:30a:c089:2b0:44ea:25b9:f2e:d23) has joined #ceph
[17:02] * wushudoin (~wushudoin@2601:646:8200:c9f0:2ab2:bdff:fe0b:a6ee) has joined #ceph
[17:04] * stefan_ (~stefan@89.207.24.152) Quit (Quit: stefan_)
[17:04] * donatas (~donatas@88-119-196-104.static.zebra.lt) has left #ceph
[17:05] * stefan_ (~stefan@89.207.24.152) has joined #ceph
[17:08] <walcubi> What's the best way to force pgs out of stuck stale+active+clean state?
[17:08] * yanzheng (~zhyan@118.116.115.254) Quit (Quit: This computer has gone to sleep)
[17:09] <walcubi> They were all last acting on the same (dead + rebuilt) osd
[17:09] * stefan_ (~stefan@89.207.24.152) has left #ceph
[17:12] * keeperandy (~textual@50-245-231-209-static.hfc.comcastbusiness.net) has joined #ceph
[17:15] <walcubi> Ooh, I have activity
[17:15] * thomnico (~thomnico@2a01:e35:8b41:120:5528:ea44:6ab4:71ed) Quit (Quit: Ex-Chat)
[17:17] * rraja (~rraja@125.16.34.66) Quit (Ping timeout: 480 seconds)
[17:27] * nass5 (~fred@l-p-dn-in-12a.lionnois.site.univ-lorraine.fr) Quit (Quit: Leaving.)
[17:27] * nass5 (~fred@l-p-dn-in-12a.lionnois.site.univ-lorraine.fr) has joined #ceph
[17:29] * ntpttr (~ntpttr@192.55.55.39) has joined #ceph
[17:35] * kristen (~kristen@134.134.139.74) has joined #ceph
[17:36] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit (Quit: WeeChat 1.5)
[17:37] * wes_dillingham (~wes_dilli@140.247.242.44) has joined #ceph
[17:38] * linuxkidd (~linuxkidd@ip70-189-202-62.lv.lv.cox.net) has joined #ceph
[17:41] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[17:44] * Dw_Sn (~Dw_Sn@00020a72.user.oftc.net) Quit (Quit: leaving)
[17:49] * sleinen (~Adium@2001:620:0:82::104) Quit (Ping timeout: 480 seconds)
[17:51] * bene2 (~bene@nat-pool-bos-t.redhat.com) has joined #ceph
[17:56] * tsg (~tgohad@192.55.54.40) has joined #ceph
[17:56] * ivve (~zed@cust-gw-11.se.zetup.net) Quit (Ping timeout: 480 seconds)
[17:59] * LiamMon (~liam.monc@electro.moncur.eu) Quit (Quit: leaving)
[18:00] * LiamMon (~liam.monc@electro.moncur.eu) has joined #ceph
[18:01] <walcubi> And restoring "lost" data.
[18:01] * raphaelsc (~raphaelsc@189.115.123.240) has joined #ceph
[18:02] <walcubi> Luckily, ceph is predictable and not only keeps the pgs on the same disk. The files I'm writing go to the exact same location as before.
[18:04] * kefu (~kefu@114.92.125.128) Quit (Ping timeout: 480 seconds)
[18:06] * borei (~dan@216.13.217.230) has joined #ceph
[18:08] * karnan (~karnan@125.16.34.66) Quit (Remote host closed the connection)
[18:19] * TMM (~hp@dhcp-077-248-009-229.chello.nl) has joined #ceph
[18:21] * kristen (~kristen@134.134.139.74) Quit (Quit: Leaving)
[18:22] * kefu (~kefu@114.92.125.128) has joined #ceph
[18:22] * sleinen (~Adium@2001:620:0:82::103) has joined #ceph
[18:24] * ade (~abradshaw@46.189.67.235) has joined #ceph
[18:33] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Quit: Bye guys! (??????????????????? ?????????)
[18:33] * kristen (~kristen@134.134.137.75) has joined #ceph
[18:36] * newbie (~kvirc@host217-114-156-249.pppoe.mark-itt.net) has joined #ceph
[18:37] * Bromine (~Dinnerbon@ab.48.caa1.ip4.static.sl-reverse.com) has joined #ceph
[18:39] * axion_joey (~oftc-webi@108.47.170.18) has joined #ceph
[18:39] <axion_joey> Hi Everyone
[18:39] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:405b:e9e5:f08e:901e) Quit (Ping timeout: 480 seconds)
[18:40] <axion_joey> has anyone had issues getting monitors up using the latest version of ceph-deploy on Centos 7?
[18:40] <axion_joey> I should say multiple monitors
[18:40] <axion_joey> If I try to configure just one monitor it's fine, but if I try to do 3 they never get in quorum
[18:43] * mykola (~Mikolaj@91.245.74.66) has joined #ceph
[18:44] * dan__ (~Daniel@2a00:1ee0:3:1337:c92d:d224:d709:500f) Quit (Quit: Leaving)
[18:47] * krypto (~krypto@G68-121-13-81.sbcis.sbc.com) has joined #ceph
[18:50] * dgurtner (~dgurtner@84-73-130-19.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[19:01] * joshd (~jdurgin@2602:30a:c089:2b0:44ea:25b9:f2e:d23) Quit (Quit: Leaving.)
[19:03] * dgurtner (~dgurtner@84-73-130-19.dclient.hispeed.ch) has joined #ceph
[19:07] * Bromine (~Dinnerbon@ab.48.caa1.ip4.static.sl-reverse.com) Quit ()
[19:11] * rakeshgm (~rakesh@106.51.28.220) has joined #ceph
[19:11] * squizzi (~squizzi@107.13.237.240) Quit (Quit: bye)
[19:13] * squizzi (~squizzi@107.13.237.240) has joined #ceph
[19:20] * Flynn (~stefan@ip-185-87-117-140.fiber.nl) has joined #ceph
[19:21] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) Quit (Quit: Leaving.)
[19:21] <Flynn> Hello all, I???m trying to install a new ceph cluster (jewel 10.2.2). I followed the quick install procedure and added two OSD???s (backed by a journal partition each on an SSD). That goes well. But, after this, the PG???s won???t get created.
[19:21] <Flynn> health HEALTH_ERR
[19:21] <Flynn> 64 pgs are stuck inactive for more than 300 seconds
[19:21] <Flynn> 64 pgs stuck inactive
[19:22] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) has joined #ceph
[19:22] <Flynn> I installed a Hammer based cluster before and never saw this happening. Can somebody point me in the right direction to get this fixed?
[19:22] <Flynn> I???m obviously doing something wrong, but can???t figure out why.
[19:23] * nils_ (~nils_@doomstreet.collins.kg) Quit (Quit: This computer has gone to sleep)
[19:23] <Flynn> $ ceph osd tree
[19:23] <Flynn> ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
[19:23] <Flynn> -1 0 root default
[19:23] <Flynn> 0 0 osd.0 up 1.00000 1.00000
[19:23] <Flynn> 1 0 osd.1 up 1.00000 1.00000
[19:23] * srk (~Siva@32.97.110.56) Quit (Quit: Leaving)
[19:24] * derjohn_mob (~aj@46.189.28.39) Quit (Ping timeout: 480 seconds)
[19:26] * kristen (~kristen@134.134.137.75) Quit (Quit: Leaving)
[19:30] * kefu (~kefu@114.92.125.128) Quit (Quit: Textual IRC Client: www.textualapp.com)
[19:33] <nwe> hello! I have a strange thing.. I using influxdb,telegraf and grafana to monitoring my ceph-cluster. I can see graph for write_bytes_sec but my read_bytes_sec is 3 days behind... any idea about it?
[19:35] <nwe> https://www.sigwait.se/grafana/ceph-grafana.png
[19:36] * krypto (~krypto@G68-121-13-81.sbcis.sbc.com) Quit (Read error: Connection reset by peer)
[19:42] * Zeis (~xolotl@static-ip-85-25-103-119.inaddr.ip-pool.com) has joined #ceph
[19:47] * ade (~abradshaw@46.189.67.235) Quit (Quit: Too sexy for his shirt)
[19:51] * squizzi_ (~squizzi@107.13.237.240) has joined #ceph
[19:52] * Lokta (~Lokta@carbon.coe.int) Quit (Ping timeout: 480 seconds)
[19:54] * ade (~abradshaw@46.189.67.235) has joined #ceph
[19:55] * doppelgrau1 (~doppelgra@132.252.235.172) Quit (Quit: Leaving.)
[19:58] * branto (~branto@178.253.167.12) Quit (Quit: Leaving.)
[19:58] * squizzi (~squizzi@107.13.237.240) Quit (Ping timeout: 480 seconds)
[20:00] * ade (~abradshaw@46.189.67.235) Quit (Remote host closed the connection)
[20:11] * salwasser1 (~Adium@a72-246-0-10.deploy.akamaitechnologies.com) has joined #ceph
[20:12] * Zeis (~xolotl@635AAAO7E.tor-irc.dnsbl.oftc.net) Quit ()
[20:12] * salwasser (~Adium@72.246.3.14) Quit (Read error: Connection reset by peer)
[20:12] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) has joined #ceph
[20:15] * joshd (~jdurgin@206.169.83.146) has joined #ceph
[20:16] * tsg (~tgohad@192.55.54.40) Quit (Remote host closed the connection)
[20:19] * TomasCZ (~TomasCZ@yes.tenlab.net) has joined #ceph
[20:19] * circ-user-lHsPC (~circuser-@154.120.74.134) has joined #ceph
[20:20] * circ-user-lHsPC is now known as sphinxx
[20:24] <sphinxx> Hello, I hope this is not out of place (i'm new to IRC and Ceph)...
[20:25] <sphinxx> Question: Is it normal for my ceph cluster CPU utilization to double within a few weeks?
[20:26] * shaunm (~shaunm@cpe-192-180-17-174.kya.res.rr.com) Quit (Ping timeout: 480 seconds)
[20:33] * doppelgrau (~doppelgra@dslb-088-072-094-200.088.072.pools.vodafone-ip.de) has joined #ceph
[20:34] * dgurtner (~dgurtner@84-73-130-19.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[20:35] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[20:35] * georgem (~Adium@206.108.127.16) has joined #ceph
[20:39] * matx (~isaxi@tor-exit.squirrel.theremailer.net) has joined #ceph
[20:40] <borei> hi all
[20:40] <borei> still can't get grip on the following
[20:41] <borei> i turned off ceph-osd on one of the node (3 nodes cluster)
[20:41] <borei> ceph supposed to mark OSD(s) down
[20:41] <borei> but it's not happening
[20:41] <borei> log full of osd.0 3863 heartbeat_check: no reply from osd.3 since back 2016-09-19 11:27:04.809046 front
[20:42] <borei> osd.0 is active
[20:42] <borei> osd.3 turned off
[20:42] <borei> why it's not marked as down
[20:42] <borei> ?
[20:46] * Jeffrey4l_ (~Jeffrey@110.252.53.161) Quit (Ping timeout: 480 seconds)
[20:47] <T1> how long has it been turned off?
[20:47] <borei> already more then 5 minutes
[20:48] <T1> there is a default 300 sec timeout from when an OSD stops respondeing to when data begins to reshuffle
[20:48] <borei> i changed it to mon osd down out interval = 5
[20:48] <borei> 5 seconds
[20:49] <T1> and restarted all MONs and OSDs afterwards?
[20:49] <borei> ??
[20:49] <borei> yes of cause
[20:49] <T1> a change in the config is not picked up unless you restart the daemons
[20:49] <borei> cluster was restarted
[20:49] <T1> .. and the changed configfile was pushed to all nodes?
[20:50] <borei> yep, all nodes
[20:50] * Jeffrey4l_ (~Jeffrey@110.252.53.161) has joined #ceph
[20:50] * andreww (~xarses@64.124.158.3) Quit (Ping timeout: 480 seconds)
[20:57] * rendar (~I@host92-69-dynamic.171-212-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[20:58] <blynch> has anyone run into an error in cbt where it doesn't dump a ceph_settings.out file? It's happening for me for rbdfio or librbdfio benchmarks in cbt, but the file is created correctly for radosbench benchmarks.
[21:01] * mattbenjamin (~mbenjamin@12.118.3.106) has joined #ceph
[21:06] * Hemanth (~hkumar_@103.228.221.171) has joined #ceph
[21:09] * matx (~isaxi@2RTAAAHJB.tor-irc.dnsbl.oftc.net) Quit ()
[21:12] * nass5 (~fred@l-p-dn-in-12a.lionnois.site.univ-lorraine.fr) Quit (Remote host closed the connection)
[21:16] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) has joined #ceph
[21:22] * ntpttr_ (~ntpttr@134.134.137.75) has joined #ceph
[21:22] * ntpttr (~ntpttr@192.55.55.39) Quit (Remote host closed the connection)
[21:23] * rendar (~I@host92-69-dynamic.171-212-r.retail.telecomitalia.it) has joined #ceph
[21:26] * hedin (~hedin@81.25.179.168) has joined #ceph
[21:26] * Realmy (~Realmy@0002243f.user.oftc.net) Quit (Quit: ZNC - http://znc.in)
[21:26] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[21:27] <hedin> Hello, is there a "best practice" for connecting ceph with windows or vmware servers, through iscsi ?
[21:28] * ntpttr_ (~ntpttr@134.134.137.75) Quit (Remote host closed the connection)
[21:29] * Realmy (~Realmy@ec2-54-172-129-45.compute-1.amazonaws.com) has joined #ceph
[21:30] * diver (~diver@95.85.8.93) has joined #ceph
[21:32] <sphinxx> Hello all, my ceph admin node's CPU utilization has doubled within a few weeks. Any idea what may be causing this?
[21:39] * johnavp1989 (~jpetrini@8.39.115.8) has joined #ceph
[21:39] <- *johnavp1989* To prove that you are human, please enter the result of 8+3
[21:48] * salwasser (~Adium@72.246.3.14) has joined #ceph
[21:48] * salwasser1 (~Adium@a72-246-0-10.deploy.akamaitechnologies.com) Quit (Read error: Connection reset by peer)
[21:49] * salwasser (~Adium@72.246.3.14) Quit ()
[21:52] * mykola (~Mikolaj@91.245.74.66) Quit (Quit: away)
[21:56] * keeperandy (~textual@50-245-231-209-static.hfc.comcastbusiness.net) Quit (Quit: Textual IRC Client: www.textualapp.com)
[21:59] <TMM> I just deleted an rbd image, but there are still rbd_data.<id> objects in rados ls
[21:59] <TMM> should I just delete those?
[22:03] * sleinen1 (~Adium@84-72-160-233.dclient.hispeed.ch) has joined #ceph
[22:07] * sleinen (~Adium@2001:620:0:82::103) Quit (Ping timeout: 480 seconds)
[22:09] * xarses (~xarses@199.16.144.11) has joined #ceph
[22:11] * diver (~diver@95.85.8.93) Quit ()
[22:11] * ntpttr (~ntpttr@192.55.54.40) has joined #ceph
[22:12] * lpabon (~quassel@nat-pool-bos-t.redhat.com) Quit (Remote host closed the connection)
[22:16] * georgem (~Adium@206.108.127.16) has left #ceph
[22:21] * qable (~Inuyasha@178-175-128-50.static.host) has joined #ceph
[22:24] * rwheeler (~rwheeler@pool-173-48-195-215.bstnma.fios.verizon.net) Quit (Quit: Leaving)
[22:26] * Hemanth (~hkumar_@103.228.221.171) Quit (Ping timeout: 480 seconds)
[22:31] <gmmaha> Hi, i am trying to run ceph services inside a fedora 24 docker container and my osd startup always fails with http://pastebin.com/cRmR3a2W
[22:31] <gmmaha> has anyone else hit this problem? Am i missing something trivial?
[22:33] * davidzlap (~Adium@2605:e000:1313:8003:1129:2f16:42d8:1acb) has joined #ceph
[22:35] * Jeffrey4l__ (~Jeffrey@110.252.65.44) has joined #ceph
[22:35] <SamYaple> gmmaha: /usr/bin/sv is part of 'runit'
[22:35] <gmmaha> SamYaple: right. And from the little searching it seems its not part of the fedora base container.
[22:36] <SamYaple> right and it isn't needed for ceph as a hard requirement either
[22:36] <SamYaple> but whatever what you are launching it needs it
[22:36] <gmmaha> ohh i forgot to say that i am using the stuff from https://github.com/ceph/ceph-docker to build the containers
[22:37] * xarses (~xarses@199.16.144.11) Quit (Ping timeout: 480 seconds)
[22:37] <SamYaple> ah yea. not a huge fan of the ceph-docker implementation myself. but I would file a bug if you are building with the instructions and it isn't working
[22:37] <SamYaple> though frankly, it just seems you need to install runit
[22:37] * raphaelsc (~raphaelsc@189.115.123.240) Quit (Remote host closed the connection)
[22:38] <gmmaha> SamYaple: got it.. Trying to install that into the base container now. Will test it out and file a bug on the same
[22:38] <gmmaha> thanks.. Wanted to just check i am not missing something uber simple
[22:38] * Jeffrey4l_ (~Jeffrey@110.252.53.161) Quit (Ping timeout: 480 seconds)
[22:42] * valeech (~valeech@166.170.31.125) has joined #ceph
[22:42] * salwasser (~Adium@c-76-118-229-231.hsd1.ma.comcast.net) has joined #ceph
[22:51] * jowilkin (~jowilkin@c-98-207-136-41.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[22:51] * qable (~Inuyasha@26XAAB1SO.tor-irc.dnsbl.oftc.net) Quit ()
[22:56] * mqciqa (~oftc-webi@24.139.73.106) has joined #ceph
[22:56] * ntpttr (~ntpttr@192.55.54.40) Quit (Remote host closed the connection)
[22:56] * ntpttr (~ntpttr@192.55.54.40) has joined #ceph
[22:56] <mqciqa> W: http://download.ceph.com/debian-jewel/dists/xenial/InRelease: Signature by key 08B73419AC32B4E966C1A330E84AC2C0460F3994 uses weak digest algorithm (SHA1) Why receive this message?
[22:57] <mqciqa> with ubuntu 16.04
[22:57] <johnavp1989> mqciqa: You can safely ignore it. It's a known issue. ubuntu is deprecating support for SHA1 so they added the warning
[22:59] * derjohn_mob (~aj@46.189.28.39) has joined #ceph
[23:00] <mqciqa> Ok thanks Johnavp1989
[23:02] * jowilkin (~jowilkin@184-23-213-254.fiber.dynamic.sonic.net) has joined #ceph
[23:05] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) Quit (Quit: treenerd_)
[23:05] * sleinen1 (~Adium@84-72-160-233.dclient.hispeed.ch) Quit (Quit: Leaving.)
[23:06] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) has joined #ceph
[23:11] * mqciqa (~oftc-webi@24.139.73.106) Quit (Remote host closed the connection)
[23:12] * johnavp1989 (~jpetrini@8.39.115.8) Quit (Ping timeout: 480 seconds)
[23:17] * valeech (~valeech@166.170.31.125) Quit (Quit: valeech)
[23:17] * wes_dillingham (~wes_dilli@140.247.242.44) Quit (Ping timeout: 480 seconds)
[23:27] * salwasser (~Adium@c-76-118-229-231.hsd1.ma.comcast.net) Quit (Quit: Leaving.)
[23:28] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) Quit (Quit: treenerd_)
[23:29] * salwasser (~Adium@2601:197:101:5cc1:98a4:2a20:d8b1:e883) has joined #ceph
[23:29] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) has joined #ceph
[23:32] * bniver (~bniver@71-9-144-29.static.oxfr.ma.charter.com) Quit (Remote host closed the connection)
[23:35] * salwasser1 (~Adium@2601:197:101:5cc1:50fe:8001:efc9:f75a) has joined #ceph
[23:37] * Racpatel (~Racpatel@2601:87:3:31e3::77ec) Quit (Quit: Leaving)
[23:37] * salwasser (~Adium@2601:197:101:5cc1:98a4:2a20:d8b1:e883) Quit (Ping timeout: 480 seconds)
[23:38] * axion_joey (~oftc-webi@108.47.170.18) Quit (Quit: Page closed)
[23:40] * Pieman (~mps@108.61.122.88) has joined #ceph
[23:43] * mistur (~yoann@kewl.mistur.org) Quit (Remote host closed the connection)
[23:44] * fsimonce (~simon@host98-71-dynamic.1-87-r.retail.telecomitalia.it) Quit (Quit: Coyote finally caught me)
[23:48] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) Quit (Quit: treenerd_)
[23:50] * treenerd_ (~gsulzberg@cpe90-146-148-47.liwest.at) has joined #ceph
[23:50] * northrup (~northrup@75-146-11-137-Nashville.hfc.comcastbusiness.net) has joined #ceph
[23:54] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) has joined #ceph
[23:54] <- *johnavp1989* To prove that you are human, please enter the result of 8+3
[23:55] * derjohn_mob (~aj@46.189.28.39) Quit (Ping timeout: 480 seconds)
[23:55] <northrup> Anyone using CephFS in production?
[23:56] * mattbenjamin (~mbenjamin@12.118.3.106) Quit (Ping timeout: 480 seconds)
[23:57] * rendar (~I@host92-69-dynamic.171-212-r.retail.telecomitalia.it) Quit (Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!)
[23:57] <borei> a bit confused with documentaion - http://docs.ceph.com/docs/jewel/rados/configuration/mon-osd-interaction/
[23:57] <borei> mon osd down out interval - is it [global] settings or [mon]
[23:58] * koma (~koma@0001c112.user.oftc.net) Quit (Ping timeout: 480 seconds)
[23:59] * mistur (~yoann@kewl.mistur.org) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.