#ceph IRC Log

Index

IRC Log for 2016-04-22

Timestamps are in GMT/BST.

[0:03] * Titin (~textual@LFbn-1-1560-65.w90-65.abo.wanadoo.fr) has joined #ceph
[0:06] * Bartek (~Bartek@dynamic-78-8-227-166.ssp.dialog.net.pl) Quit (Ping timeout: 480 seconds)
[0:06] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[0:07] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[0:08] * haomaiwang (~haomaiwan@2600:1004:b069:6936:5957:11e1:cfbe:8dc7) has joined #ceph
[0:10] * Skaag (~lunix@65.200.54.234) Quit (Quit: Leaving.)
[0:12] <Kupo1> What is the difference between 'pg 3.261b8805' in 'ceph osd map $pool' and 'pg 3.261' from 'ceph health detail'?
[0:14] * fsimonce (~simon@host201-70-dynamic.26-79-r.retail.telecomitalia.it) Quit (Quit: Coyote finally caught me)
[0:14] * Bartek (~Bartek@dynamic-78-8-227-166.ssp.dialog.net.pl) has joined #ceph
[0:14] * Skaag (~lunix@65.200.54.234) has joined #ceph
[0:16] * haomaiwang (~haomaiwan@2600:1004:b069:6936:5957:11e1:cfbe:8dc7) Quit (Ping timeout: 480 seconds)
[0:20] * ibravo (~ibravo@72.83.69.64) has joined #ceph
[0:21] <ibravo> currently installing http://download.ceph.com/rpm-jewel/el7/ and is complaining about "Package does not match intended download"
[0:22] * lmb (~Lars@74.203.127.5) Quit (Ping timeout: 480 seconds)
[0:22] <PoRNo-MoRoZ> 0.25%
[0:22] * Skaag (~lunix@65.200.54.234) Quit (Ping timeout: 480 seconds)
[0:22] * Titin (~textual@LFbn-1-1560-65.w90-65.abo.wanadoo.fr) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[0:23] * mattbenjamin (~mbenjamin@50.59.37.123) has joined #ceph
[0:23] * sudocat (~dibarra@192.185.1.20) has joined #ceph
[0:28] * Zyn (~Jourei@06SAABJ6K.tor-irc.dnsbl.oftc.net) Quit ()
[0:28] * Helleshin (~Aethis@hessel3.torservers.net) has joined #ceph
[0:37] * thomnico (~thomnico@12.237.105.2) Quit (Ping timeout: 480 seconds)
[0:38] * Bartek (~Bartek@dynamic-78-8-227-166.ssp.dialog.net.pl) Quit (Remote host closed the connection)
[0:38] <PoRNo-MoRoZ> okay it stopped
[0:39] <PoRNo-MoRoZ> 2484 active+clean
[0:39] <PoRNo-MoRoZ> 66 down+remapped+peering
[0:39] <PoRNo-MoRoZ> 7 down+peering
[0:39] <PoRNo-MoRoZ> 3 active+recovering+undersized+degraded+remapped
[0:40] <PoRNo-MoRoZ> i still need that faulty osd ?
[0:40] * dneary (~dneary@50-206-118-3-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[0:50] * Skaag (~lunix@65.200.54.234) has joined #ceph
[0:51] * Skaag (~lunix@65.200.54.234) Quit ()
[0:52] <PoRNo-MoRoZ> osd/PGLog.cc: 382: FAILED assert(objiter->second->version > last_divergent_update)
[0:52] <PoRNo-MoRoZ> problematic osd ><
[0:55] * haomaiwang (~haomaiwan@2600:1004:b069:6936:852b:c492:658b:4cc6) has joined #ceph
[0:58] * Helleshin (~Aethis@4MJAAEDZL.tor-irc.dnsbl.oftc.net) Quit ()
[0:58] * MatthewH12 (~Guest1390@62.210.74.137) has joined #ceph
[1:01] <PoRNo-MoRoZ> 2484 active+clean
[1:01] <PoRNo-MoRoZ> 73 down+remapped+peering
[1:01] <PoRNo-MoRoZ> 2 active+recovering+undersized+degraded+remapped
[1:01] <PoRNo-MoRoZ> 1 active+recovery_wait+undersized+degraded+remapped
[1:01] <PoRNo-MoRoZ> now this
[1:01] <PoRNo-MoRoZ> i just tried to start that osd
[1:01] <PoRNo-MoRoZ> and reweighted it to 0 again
[1:02] <PoRNo-MoRoZ> should i crush reweight that osd to 0 ?
[1:03] * haomaiwang (~haomaiwan@2600:1004:b069:6936:852b:c492:658b:4cc6) Quit (Ping timeout: 480 seconds)
[1:03] * vata (~vata@cable-21.246.173-197.electronicbox.net) has joined #ceph
[1:06] <PoRNo-MoRoZ> oh
[1:06] <PoRNo-MoRoZ> u got me waited
[1:06] <PoRNo-MoRoZ> now you all gone
[1:06] <PoRNo-MoRoZ> :D
[1:09] * i_m (~ivan.miro@88.206.104.168) Quit (Ping timeout: 480 seconds)
[1:09] * mattbenjamin (~mbenjamin@50.59.37.123) Quit (Remote host closed the connection)
[1:10] * mattbenjamin (~mbenjamin@50.59.37.123) has joined #ceph
[1:10] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[1:18] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[1:22] * sudocat (~dibarra@192.185.1.20) Quit (Ping timeout: 480 seconds)
[1:25] * BrianA (~BrianA@fw-rw.shutterfly.com) has joined #ceph
[1:28] * MatthewH12 (~Guest1390@4MJAAEDZ4.tor-irc.dnsbl.oftc.net) Quit ()
[1:28] * hgjhgjh1 (~Enikma@4MJAAED0T.tor-irc.dnsbl.oftc.net) has joined #ceph
[1:31] * BrianA2 (~BrianA@fw-rw.shutterfly.com) Quit (Ping timeout: 480 seconds)
[1:34] * codice (~toodles@75-128-34-237.static.mtpk.ca.charter.com) Quit (Remote host closed the connection)
[1:34] * mattbenjamin (~mbenjamin@50.59.37.123) Quit (Ping timeout: 480 seconds)
[1:36] * oms101 (~oms101@p20030057EA013F00C6D987FFFE4339A1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[1:36] * haplo37 (~haplo37@107.190.32.70) has joined #ceph
[1:38] * MentalRay (~MentalRay@107.171.161.165) has joined #ceph
[1:41] * The1_ (~the_one@87.104.212.66) has joined #ceph
[1:43] <PoRNo-MoRoZ> xcezzz TMM ?
[1:45] * oms101 (~oms101@2003:57:ea01:800:c6d9:87ff:fe43:39a1) has joined #ceph
[1:45] * ZyTer_ (~ZyTer@ghostbusters.apinnet.fr) has joined #ceph
[1:46] * treenerd_ (~Gerhard@85.193.140.98) has joined #ceph
[1:46] * wogri_ (~wolf@nix.wogri.at) has joined #ceph
[1:47] * Mosibi (~Mosibi@77.37.12.119) Quit (Read error: Connection reset by peer)
[1:47] * ZyTer (~ZyTer@ghostbusters.apinnet.fr) Quit (Ping timeout: 480 seconds)
[1:47] * Nixx_ (~quassel@bulbasaur.sjorsgielen.nl) has joined #ceph
[1:48] * [arx] (~arx@six.mac-anu.org) Quit (Ping timeout: 480 seconds)
[1:48] * [arx] (~arx@the.kittypla.net) has joined #ceph
[1:48] * treenerd (~Gerhard@85.193.140.98) Quit (Ping timeout: 480 seconds)
[1:48] * T1 (~the_one@87.104.212.66) Quit (Ping timeout: 480 seconds)
[1:48] * wogri (~wolf@nix.wogri.at) Quit (Ping timeout: 480 seconds)
[1:48] * babilen (~babilen@babilen.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:49] * Nixx (~quassel@bulbasaur.sjorsgielen.nl) Quit (Ping timeout: 480 seconds)
[1:51] * Mosibi (~Mosibi@dld.unixguru.nl) has joined #ceph
[1:56] <PoRNo-MoRoZ> guys, can i create dummy osd and put pgs in it ?
[1:56] <PoRNo-MoRoZ> 'reverse hide'
[1:58] * hgjhgjh1 (~Enikma@4MJAAED0T.tor-irc.dnsbl.oftc.net) Quit ()
[1:58] * djidis__ (~LorenXo@4.tor.exit.babylon.network) has joined #ceph
[2:05] * billwebb (~billwebb@50.59.37.123) has joined #ceph
[2:05] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[2:10] <motk> not really
[2:10] <motk> 'undersized' is a clue
[2:10] <motk> you can't satisfy your crushmap as it is
[2:16] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[2:17] * Lea (~LeaChim@host86-176-19-208.range86-176.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:19] * dalgaaf (uid15138@id-15138.ealing.irccloud.com) Quit (Quit: Connection closed for inactivity)
[2:24] * xarses (~xarses@64.124.158.100) Quit (Ping timeout: 480 seconds)
[2:26] * loth1 (~Kupo@85.17.130.33) has joined #ceph
[2:28] * djidis__ (~LorenXo@76GAAEQQ2.tor-irc.dnsbl.oftc.net) Quit ()
[2:28] * Wijk (~Bromine@193.90.12.89) has joined #ceph
[2:30] * billwebb (~billwebb@50.59.37.123) Quit (Quit: billwebb)
[2:30] * loth (~Kupo@85.17.130.33) Quit (Ping timeout: 480 seconds)
[2:34] * Eduardo_ (~Eduardo@189.196.54.77.rev.vodafone.pt) has joined #ceph
[2:34] * billwebb (~billwebb@50.59.37.123) has joined #ceph
[2:35] * huangjun (~kvirc@113.57.168.154) has joined #ceph
[2:35] <Eduardo_> Hi everyone. I'm going around in circles, dunno what else to do. I have a Centos 7 nodes Ceph configuration, and after doing osd create, all OSDs are always down/out
[2:36] <Eduardo_> If I try to star the process on the nodes, it says that the OSd processes entered a failed state
[2:36] <Eduardo_> Anyone knows what can be the issue here?
[2:41] <Eduardo_> process status says "code=exited, status=1/FAILURE"
[2:43] <PoRNo-MoRoZ> motk thanks !
[2:45] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) has joined #ceph
[2:45] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) Quit (Remote host closed the connection)
[2:45] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) has joined #ceph
[2:48] <PoRNo-MoRoZ> Eduardo_
[2:48] <PoRNo-MoRoZ> /var/log/ceph
[2:49] <PoRNo-MoRoZ> :)
[2:50] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[2:51] <PoRNo-MoRoZ> also when i'm pushing to osds i got freezez, pushing ctrl+c and got this
[2:51] <PoRNo-MoRoZ> osd.3: Error ENXIO: problem getting command descriptions from osd.3
[2:51] <PoRNo-MoRoZ> ^Cosd.5: Error EINTR: problem getting command descriptions from osd.5
[2:54] * deepthi (~deepthi@115.118.24.250) has joined #ceph
[2:55] * brians__ (~brian@80.111.114.175) has joined #ceph
[2:56] * brians__ (~brian@80.111.114.175) Quit (Max SendQ exceeded)
[2:56] <PoRNo-MoRoZ> that osds or down or weighted 0
[2:57] * brians__ (~brian@80.111.114.175) has joined #ceph
[2:58] * Wijk (~Bromine@4MJAAED17.tor-irc.dnsbl.oftc.net) Quit ()
[3:00] * MentalRay (~MentalRay@107.171.161.165) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[3:00] <Eduardo_> PoRNo-MoRoZ, on admin log there is no new entry, just an old one complaining "unable to open OSD superblock on /var/lib/ceph/osd/ceph0"
[3:01] * MentalRay (~MentalRay@107.171.161.165) has joined #ceph
[3:01] <Eduardo_> On OSD nodes, there is a log per OSD but only shows pids
[3:01] * brians_ (~brian@80.111.114.175) Quit (Ping timeout: 480 seconds)
[3:04] * csoukup (~csoukup@2605:a601:9c8:6b00:ec0c:d83f:5c09:d1e9) has joined #ceph
[3:04] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[3:07] * deepthi (~deepthi@115.118.24.250) Quit (Quit: Leaving)
[3:08] * JustEra (~JustEra@my83-216-95-243.cust.relish.net) has joined #ceph
[3:08] * JustEra (~JustEra@my83-216-95-243.cust.relish.net) Quit ()
[3:09] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[3:09] * lcurtis_ (~lcurtis@47.19.105.250) Quit (Quit: Ex-Chat)
[3:12] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[3:12] * yanzheng (~zhyan@125.70.21.113) has joined #ceph
[3:14] * csoukup (~csoukup@2605:a601:9c8:6b00:ec0c:d83f:5c09:d1e9) Quit (Ping timeout: 480 seconds)
[3:16] * billwebb (~billwebb@50.59.37.123) Quit (Read error: No route to host)
[3:20] <Eduardo_> Also, monitor logs just claim it detects 0 OSDs
[3:21] * csoukup (~csoukup@2605:a601:9c8:6b00:ec0c:d83f:5c09:d1e9) has joined #ceph
[3:23] * mattbenjamin (~mbenjamin@12.31.71.58) has joined #ceph
[3:25] * DG1 (~Adium@inet-hqmc01-o.oracle.com) Quit (Remote host closed the connection)
[3:25] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[3:25] * Mika_c (~quassel@122.146.93.152) has joined #ceph
[3:28] * qable (~Pieman@exit1.ipredator.se) has joined #ceph
[3:32] * timfreund (~tim@ec2-54-209-140-45.compute-1.amazonaws.com) has left #ceph
[3:33] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[3:34] * BrianA (~BrianA@fw-rw.shutterfly.com) Quit (Read error: Connection reset by peer)
[3:38] * brians_ (~brian@80.111.114.175) has joined #ceph
[3:44] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[3:44] * brians__ (~brian@80.111.114.175) Quit (Ping timeout: 480 seconds)
[3:45] * mattbenjamin (~mbenjamin@12.31.71.58) Quit (Quit: Leaving.)
[3:46] * ibravo (~ibravo@72.83.69.64) Quit (Quit: Quitting channel)
[3:48] <huangjun> i got message "got xxx + x + xxx byte message.. ABORTED" and "reader bad tag 0"
[3:48] <huangjun> does this mean the message is corrupt?
[3:50] * EinstCra_ (~EinstCraz@58.247.119.250) has joined #ceph
[3:53] * derjohn_mobi (~aj@x4db0c2f6.dyn.telefonica.de) has joined #ceph
[3:53] * derjohn_mob (~aj@x590d52c3.dyn.telefonica.de) Quit (Read error: Connection reset by peer)
[3:54] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[3:58] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Ping timeout: 480 seconds)
[3:58] * qable (~Pieman@4MJAAED3T.tor-irc.dnsbl.oftc.net) Quit ()
[4:03] * zhaochao (~zhaochao@125.39.112.5) has joined #ceph
[4:12] * dneary (~dneary@96.95.216.225) has joined #ceph
[4:17] <TMM> PoRNo-MoRoZ, you have that osd on a weight of 0, right? The failed one?
[4:21] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[4:26] * Vacuum__ (~Vacuum@i59F797B4.versanet.de) has joined #ceph
[4:28] * VampiricPadraig (~Bonzaii@06SAABKD8.tor-irc.dnsbl.oftc.net) has joined #ceph
[4:28] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) Quit (Remote host closed the connection)
[4:29] * daiver (~daiver@cpe-98-26-71-226.nc.res.rr.com) has joined #ceph
[4:32] * Vacuum_ (~Vacuum@88.130.221.129) Quit (Ping timeout: 480 seconds)
[4:35] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) Quit (Quit: valeech)
[4:37] * daiver (~daiver@cpe-98-26-71-226.nc.res.rr.com) Quit (Ping timeout: 480 seconds)
[4:37] * mdxi (~mdxi@li925-141.members.linode.com) has joined #ceph
[4:37] <PoRNo-MoRoZ> yep
[4:37] <PoRNo-MoRoZ> alrigh i managed to make a script that lists for problematic pgs and looks for it in failed osd
[4:38] <PoRNo-MoRoZ> exporting now
[4:40] * Vacuum_ (~Vacuum@88.130.194.198) has joined #ceph
[4:42] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit (Quit: WeeChat 1.4)
[4:43] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[4:43] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit ()
[4:44] <TMM> PoRNo-MoRoZ, the undersized pgs are probably because the weight is 0 on one of your osds
[4:44] <PoRNo-MoRoZ> what's the purpose of folder 'snap_NNNNNNNN' inside osd folder ?
[4:44] * fdmanana (~fdmanana@74.203.127.5) has joined #ceph
[4:44] <TMM> PoRNo-MoRoZ, and you don't have enough osds now to host all copies of all pgs
[4:45] <TMM> you can only have so many pgs on an osd
[4:45] <TMM> if you have more pgs in your pools than you have osds for you get undersized as well
[4:45] <TMM> do you still have lost objects now?
[4:45] <PoRNo-MoRoZ> yep
[4:46] <PoRNo-MoRoZ> still not 100% done
[4:46] <PoRNo-MoRoZ> 0.77%
[4:46] <PoRNo-MoRoZ> ><
[4:46] <PoRNo-MoRoZ> active+remapped+backfill_toofull
[4:46] <PoRNo-MoRoZ> toolfull
[4:46] <PoRNo-MoRoZ> toofull
[4:46] <PoRNo-MoRoZ> how can i temporary raise ?
[4:46] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[4:46] <PoRNo-MoRoZ> ceph tell osd.* injectargs '--osd-backfill-full-ratio 0.95'
[4:46] <TMM> you can change the nearfull ratio
[4:46] <PoRNo-MoRoZ> will it works ?
[4:46] * Vacuum___ (~Vacuum@88.130.192.5) has joined #ceph
[4:47] * Vacuum__ (~Vacuum@i59F797B4.versanet.de) Quit (Ping timeout: 480 seconds)
[4:48] <PoRNo-MoRoZ> TMM what's the purpose of 'snap_XXXXXX' folder ?
[4:49] <PoRNo-MoRoZ> actually
[4:49] <PoRNo-MoRoZ> if i disabled it
[4:49] <PoRNo-MoRoZ> :)
[4:49] <PoRNo-MoRoZ> should it exists ?
[4:49] <TMM> I don't know, I haven't seen those. It may be btrfs snaps
[4:49] <PoRNo-MoRoZ> filestore_btrfs_snap = false
[4:49] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit ()
[4:49] <PoRNo-MoRoZ> hm
[4:50] <TMM> I've searched one of my osds for snap_ directories, I don't have them
[4:50] * dneary (~dneary@96.95.216.225) Quit (Ping timeout: 480 seconds)
[4:50] <TMM> maybe someone else can tell you
[4:51] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[4:53] * Vacuum_ (~Vacuum@88.130.194.198) Quit (Ping timeout: 480 seconds)
[4:54] <PoRNo-MoRoZ> 'near' full not means 'frozen' ?
[4:54] <PoRNo-MoRoZ> 'near full' not means 'frozen' ?
[4:56] * toastydeath (~toast@pool-71-255-253-39.washdc.fios.verizon.net) has joined #ceph
[4:57] * vbellur (~vijay@2601:18f:700:55b0:5e51:4fff:fee8:6a5c) has joined #ceph
[4:57] * mdxi (~mdxi@li925-141.members.linode.com) Quit (Quit: leaving)
[4:57] * mdxi (~mdxi@li925-141.members.linode.com) has joined #ceph
[4:58] * VampiricPadraig (~Bonzaii@06SAABKD8.tor-irc.dnsbl.oftc.net) Quit ()
[4:58] <TMM> I don't think that ceph will backfill past nearfull
[4:58] * loft (~Vale@tor1e1.privacyfoundation.ch) has joined #ceph
[4:59] <TMM> maybe have a quick look by running df on all your hosts to see if you have an osd disk that's actually entirely full
[5:00] * efirs (~firs@c-50-185-70-125.hsd1.ca.comcast.net) has joined #ceph
[5:01] * nils__ (~nils@port-19141.pppoe.wtnet.de) Quit (Ping timeout: 480 seconds)
[5:02] * toastyde1th (~toast@pool-71-255-253-39.washdc.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[5:06] * nils__ (~nils@port-19141.pppoe.wtnet.de) has joined #ceph
[5:06] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) Quit (Ping timeout: 480 seconds)
[5:07] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit (Quit: WeeChat 1.4)
[5:09] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[5:10] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit ()
[5:13] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[5:14] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit ()
[5:14] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[5:18] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[5:18] * valeech (~valeech@ip-64-134-185-50.public.wayport.net) has joined #ceph
[5:19] * overclk (~quassel@117.202.103.68) has joined #ceph
[5:20] * Vacuum_ (~Vacuum@88.130.204.120) has joined #ceph
[5:23] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[5:27] * Vacuum___ (~Vacuum@88.130.192.5) Quit (Ping timeout: 480 seconds)
[5:28] * loft (~Vale@76GAAEQUH.tor-irc.dnsbl.oftc.net) Quit ()
[5:28] * Behedwin (~w0lfeh@tor-exit0-readme.dfri.se) has joined #ceph
[5:30] * haplo37 (~haplo37@107.190.32.70) Quit (Remote host closed the connection)
[5:33] * natarej (~natarej@CPE-101-181-149-113.lnse5.cha.bigpond.net.au) Quit (Read error: Connection reset by peer)
[5:35] <PoRNo-MoRoZ> okay i'm importing now
[5:35] * overclk_ (~quassel@117.202.103.68) has joined #ceph
[5:36] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) has joined #ceph
[5:39] * overclk (~quassel@117.202.103.68) Quit (Ping timeout: 480 seconds)
[5:42] * fdmanana (~fdmanana@74.203.127.5) Quit (Ping timeout: 480 seconds)
[5:46] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[5:58] * Behedwin (~w0lfeh@6AGAAA63L.tor-irc.dnsbl.oftc.net) Quit ()
[5:58] * dusti (~datagutt@politkovskaja.torservers.net) has joined #ceph
[5:58] * overclk_ is now known as overclk
[6:00] * vikhyat (~vumrao@121.244.87.116) has joined #ceph
[6:02] * nils___ (~nils@port-54975.pppoe.wtnet.de) has joined #ceph
[6:04] * kefu (~kefu@183.193.162.205) has joined #ceph
[6:07] * Vacuum__ (~Vacuum@i59F79C11.versanet.de) has joined #ceph
[6:07] * nils__ (~nils@port-19141.pppoe.wtnet.de) Quit (Ping timeout: 480 seconds)
[6:08] <PoRNo-MoRoZ> 74/130 GB imported ..
[6:08] <PoRNo-MoRoZ> oh god
[6:09] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[6:09] * yatin (~yatin@125.99.244.76) has joined #ceph
[6:10] <PoRNo-MoRoZ> btw
[6:10] <PoRNo-MoRoZ> remember my synology ?
[6:10] <PoRNo-MoRoZ> it's the only node that works with /etc/init.d/ceph okay
[6:10] <PoRNo-MoRoZ> it can restart per-node
[6:10] <PoRNo-MoRoZ> /etc/init.d/ceph restart osd.x
[6:10] <PoRNo-MoRoZ> somehow other nodes cannot
[6:11] <PoRNo-MoRoZ> it was clean debian 8 netinst
[6:11] <PoRNo-MoRoZ> with basic system and ssh
[6:11] <PoRNo-MoRoZ> the rest can restart only all ceph services
[6:12] * valeech (~valeech@ip-64-134-185-50.public.wayport.net) Quit (Quit: valeech)
[6:12] * yk (~yatin@216.207.42.140) has joined #ceph
[6:13] * Vacuum_ (~Vacuum@88.130.204.120) Quit (Ping timeout: 480 seconds)
[6:16] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Quit: Leaving.)
[6:17] * MentalRay (~MentalRay@107.171.161.165) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[6:18] * overclk_ (~quassel@117.202.100.194) has joined #ceph
[6:19] * yatin (~yatin@125.99.244.76) Quit (Ping timeout: 480 seconds)
[6:19] * mnathani (~mnathani_@192-0-149-228.cpe.teksavvy.com) has joined #ceph
[6:19] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) has joined #ceph
[6:21] * overclk (~quassel@117.202.103.68) Quit (Ping timeout: 480 seconds)
[6:26] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[6:28] * dusti (~datagutt@76GAAEQVM.tor-irc.dnsbl.oftc.net) Quit ()
[6:30] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) has joined #ceph
[6:33] * rdas (~rdas@121.244.87.116) has joined #ceph
[6:33] * nils__ (~nils@port-19252.pppoe.wtnet.de) has joined #ceph
[6:34] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[6:37] * nils___ (~nils@port-54975.pppoe.wtnet.de) Quit (Ping timeout: 480 seconds)
[6:38] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) Quit (Ping timeout: 480 seconds)
[6:43] <PoRNo-MoRoZ> okay i imported and started os
[6:43] <PoRNo-MoRoZ> okay i imported and started osd
[6:43] <PoRNo-MoRoZ> how do i know it will work ?
[6:43] <PoRNo-MoRoZ> oh shi
[6:43] <PoRNo-MoRoZ> TMM
[6:43] <PoRNo-MoRoZ> i forgot
[6:43] <PoRNo-MoRoZ> should i remove entirely broken osd from crush map ?
[6:43] <PoRNo-MoRoZ> ceph osd rm ..
[6:44] <PoRNo-MoRoZ> atm i see no decreasing of 'down' pgs
[6:44] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[6:45] <PoRNo-MoRoZ> at this moment there is no activity in logs
[6:45] <PoRNo-MoRoZ> but cluster still rebalancing
[6:45] <PoRNo-MoRoZ> due 'low space'
[6:45] <PoRNo-MoRoZ> i added some disks back
[6:45] * kefu_ (~kefu@114.92.122.74) has joined #ceph
[6:46] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Remote host closed the connection)
[6:46] * haomaiwang (~haomaiwan@2600:1004:b069:6936:bc63:67a:1ea7:42d8) has joined #ceph
[6:48] <TMM> PoRNo-MoRoZ, yeah, you want to rm it, remove its authkeys
[6:49] <TMM> PoRNo-MoRoZ, then zap the disk, and create a new osd
[6:50] * Vacuum_ (~Vacuum@88.130.218.13) has joined #ceph
[6:50] <TMM> PoRNo-MoRoZ, but only if your cluster is HEALTHY
[6:50] <PoRNo-MoRoZ> only after that ?
[6:50] <PoRNo-MoRoZ> can i remove it from crush, but don't format it ?
[6:50] * kefu (~kefu@183.193.162.205) Quit (Ping timeout: 480 seconds)
[6:53] <PoRNo-MoRoZ> i mean
[6:53] <PoRNo-MoRoZ> if it's still in crush
[6:53] <PoRNo-MoRoZ> can cluster repair himself ?
[6:54] <PoRNo-MoRoZ> with osd with imported pgs
[6:54] <PoRNo-MoRoZ> and weight 0
[6:55] * arthurh (~arthurh@38.101.34.128) Quit (Read error: No route to host)
[6:57] * Vacuum__ (~Vacuum@i59F79C11.versanet.de) Quit (Ping timeout: 480 seconds)
[6:57] * Vacuum__ (~Vacuum@88.130.212.154) has joined #ceph
[6:58] * flisky (~Thunderbi@36.110.40.24) has joined #ceph
[6:59] * huangjun|2 (~kvirc@113.57.168.154) has joined #ceph
[7:03] * TMM (~hp@178-84-46-106.dynamic.upc.nl) Quit (Ping timeout: 480 seconds)
[7:04] * Vacuum_ (~Vacuum@88.130.218.13) Quit (Ping timeout: 480 seconds)
[7:05] * huangjun (~kvirc@113.57.168.154) Quit (Ping timeout: 480 seconds)
[7:10] * TMM (~hp@178-84-46-106.dynamic.upc.nl) has joined #ceph
[7:11] * Inverness (~xul@Relay-J.tor-exit.network) has joined #ceph
[7:12] * haomaiwa_ (~haomaiwan@2600:1004:b069:6936:bc63:67a:1ea7:42d8) has joined #ceph
[7:12] * Nacer (~Nacer@vir78-1-82-232-38-190.fbx.proxad.net) has joined #ceph
[7:17] * haomaiwang (~haomaiwan@2600:1004:b069:6936:bc63:67a:1ea7:42d8) Quit (Ping timeout: 480 seconds)
[7:19] * Vacuum_ (~Vacuum@88.130.196.215) has joined #ceph
[7:21] * Vacuum__ (~Vacuum@88.130.212.154) Quit (Ping timeout: 480 seconds)
[7:21] * Nacer (~Nacer@vir78-1-82-232-38-190.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[7:22] * yatin (~yatin@125.99.244.76) has joined #ceph
[7:23] * flisky1 (~Thunderbi@36.110.40.23) has joined #ceph
[7:23] * flisky (~Thunderbi@36.110.40.24) Quit (Read error: Connection reset by peer)
[7:23] * flisky1 is now known as flisky
[7:29] * yk (~yatin@216.207.42.140) Quit (Ping timeout: 480 seconds)
[7:31] * daiver (~daiver@cpe-98-26-71-226.nc.res.rr.com) has joined #ceph
[7:31] * Vacuum__ (~Vacuum@88.130.202.140) has joined #ceph
[7:33] * yatin (~yatin@125.99.244.76) Quit (Remote host closed the connection)
[7:33] * redf (~red@80-108-89-163.cable.dynamic.surfer.at) has joined #ceph
[7:34] * haomaiwa_ (~haomaiwan@2600:1004:b069:6936:bc63:67a:1ea7:42d8) Quit (Remote host closed the connection)
[7:34] * Vacuum_ (~Vacuum@88.130.196.215) Quit (Ping timeout: 480 seconds)
[7:35] * ninkotech_ (~duplo@static-84-242-87-186.net.upcbroadband.cz) has joined #ceph
[7:37] * redf_ (~red@80-108-89-163.cable.dynamic.surfer.at) Quit (Ping timeout: 480 seconds)
[7:38] * The_Ball (~pi@20.92-221-43.customer.lyse.net) Quit (Ping timeout: 480 seconds)
[7:39] * ninkotech (~duplo@static-84-242-87-186.net.upcbroadband.cz) Quit (Ping timeout: 480 seconds)
[7:40] * daiver (~daiver@cpe-98-26-71-226.nc.res.rr.com) Quit (Ping timeout: 480 seconds)
[7:41] * Inverness (~xul@6AGAAA676.tor-irc.dnsbl.oftc.net) Quit ()
[7:41] * matx (~curtis864@politkovskaja.torservers.net) has joined #ceph
[7:47] * The_Ball (~pi@20.92-221-43.customer.lyse.net) has joined #ceph
[7:53] * Vacuum_ (~Vacuum@88.130.208.236) has joined #ceph
[7:55] * Vacuum__ (~Vacuum@88.130.202.140) Quit (Ping timeout: 480 seconds)
[7:57] * dneary (~dneary@96.95.216.225) has joined #ceph
[7:58] * Vacuum__ (~Vacuum@88.130.210.141) has joined #ceph
[7:59] * rotbeard (~redbeard@2a02:908:df18:b980:6267:20ff:feb7:c20) has joined #ceph
[8:00] * branto (~branto@nat-pool-brq-t.redhat.com) has joined #ceph
[8:01] * dgurtner (~dgurtner@178.197.235.112) has joined #ceph
[8:01] * Vacuum_ (~Vacuum@88.130.208.236) Quit (Ping timeout: 480 seconds)
[8:04] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) Quit (Ping timeout: 480 seconds)
[8:07] * Vacuum__ (~Vacuum@88.130.210.141) Quit (Ping timeout: 480 seconds)
[8:08] <PoRNo-MoRoZ> 0.06%
[8:08] <PoRNo-MoRoZ> oh god
[8:09] <PoRNo-MoRoZ> btw glitchy osd still present in crush map
[8:09] <PoRNo-MoRoZ> ah
[8:09] <PoRNo-MoRoZ> wait a moment
[8:09] <PoRNo-MoRoZ> i didn't deleted it yet
[8:09] <PoRNo-MoRoZ> i'm waiting
[8:09] * neurodrone (~neurodron@162.243.191.67) Quit (Ping timeout: 480 seconds)
[8:10] <PoRNo-MoRoZ> okay now it stops
[8:10] <PoRNo-MoRoZ> looks like
[8:10] <PoRNo-MoRoZ> should i kill clients that still using that cluster ?
[8:10] * Vacuum_ (~Vacuum@i59F7954B.versanet.de) has joined #ceph
[8:10] <PoRNo-MoRoZ> 2016-04-22 09:09:27.472587 osd.6 10.30.50.41:6808/1182 2260 : cluster [ERR] 1.85 has 1 objects unfound and apparently lost
[8:10] <PoRNo-MoRoZ> 2016-04-22 09:09:28.630025 osd.6 10.30.50.41:6808/1182 2261 : cluster [ERR] 1.85 has 1 objects unfound and apparently lost
[8:10] <PoRNo-MoRoZ> 2016-04-22 09:09:27.475275 osd.16 10.30.50.41:6856/1765 1484 : cluster [ERR] 1.2f0 has 1 objects unfound and apparently lost
[8:10] <PoRNo-MoRoZ> 2016-04-22 09:09:27.475803 osd.16 10.30.50.41:6856/1765 1485 : cluster [ERR] 1.7d has 2 objects unfound and apparently lost
[8:10] <PoRNo-MoRoZ> 2016-04-22 09:09:28.649929 osd.16 10.30.50.41:6856/1765 1486 : cluster [ERR] 1.2f0 has 1 objects unfound and apparently lost
[8:10] <PoRNo-MoRoZ> 2016-04-22 09:09:28.650701 osd.16 10.30.50.41:6856/1765 1487 : cluster [ERR] 1.7d has 2 objects unfound and apparently lost
[8:10] <PoRNo-MoRoZ> i can see this in logs
[8:11] <PoRNo-MoRoZ> but it's not using my osd with imported data
[8:11] * matx (~curtis864@6AGAAA686.tor-irc.dnsbl.oftc.net) Quit ()
[8:12] <PoRNo-MoRoZ> TMM
[8:12] <PoRNo-MoRoZ> should i add this dummy-osd to POOL ?
[8:16] <PoRNo-MoRoZ> wow
[8:16] <PoRNo-MoRoZ> i managed to remove it from crush map
[8:16] <PoRNo-MoRoZ> and now it recovering
[8:16] <PoRNo-MoRoZ> looks like
[8:16] <PoRNo-MoRoZ> wonderful
[8:18] * wogri_ (~wolf@nix.wogri.at) Quit (Quit: Lost terminal)
[8:18] * wogri (~wolf@nix.wogri.at) has joined #ceph
[8:18] <PoRNo-MoRoZ> okay got 2 incomplete
[8:18] <PoRNo-MoRoZ> rest backfilling atm
[8:18] <PoRNo-MoRoZ> waiting ..
[8:18] * Vacuum_ (~Vacuum@i59F7954B.versanet.de) Quit (Ping timeout: 480 seconds)
[8:20] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) has joined #ceph
[8:26] * shylesh__ (~shylesh@45.124.227.15) has joined #ceph
[8:26] <PoRNo-MoRoZ> vms unfrozen btw
[8:26] <PoRNo-MoRoZ> after like 12 hours or smth
[8:28] * dugravot6 (~dugravot6@dn-infra-04.lionnois.site.univ-lorraine.fr) has joined #ceph
[8:32] * Titin (~textual@LFbn-1-1560-65.w90-65.abo.wanadoo.fr) has joined #ceph
[8:34] * babilen (~babilen@babilen.user.oftc.net) has joined #ceph
[8:35] <TMM> PoRNo-MoRoZ, good!
[8:36] <PoRNo-MoRoZ> still got 4 unfounds
[8:36] <PoRNo-MoRoZ> but recovery still in progress
[8:36] <TMM> same thing applies, wait, wait, wait, I know it's hard :)
[8:36] <motk> your crush map should be satisfiable if you lose an osd
[8:37] <motk> easier said than done but that's how it works
[8:37] <PoRNo-MoRoZ> yep ><
[8:37] <TMM> motk, he has size 2, min size 1 on only 2 nodes
[8:37] <PoRNo-MoRoZ> :DD
[8:37] * ivancich (~ivancich@aa2.linuxbox.com) Quit (Ping timeout: 480 seconds)
[8:37] <PoRNo-MoRoZ> no i got 3 nodes
[8:37] <motk> http://dachary.org/?p=2562
[8:39] * derjohn_mobi (~aj@x4db0c2f6.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[8:39] <TMM> oh, that's a little better I suppose :P
[8:39] * Hemanth (~hkumar_@121.244.87.117) has joined #ceph
[8:40] * shylesh__ (~shylesh@45.124.227.15) Quit (Ping timeout: 480 seconds)
[8:41] * visored (~Azerothia@195.22.126.119) has joined #ceph
[8:43] * shohn (~shohn@dslb-092-078-051-109.092.078.pools.vodafone-ip.de) has joined #ceph
[8:46] <PoRNo-MoRoZ> okay
[8:46] <PoRNo-MoRoZ> it stops
[8:46] <PoRNo-MoRoZ> 2557 active+clean
[8:46] <PoRNo-MoRoZ> 3 active+recovering+undersized+degraded+remapped
[8:46] <PoRNo-MoRoZ> recovery 1686/7735910 objects degraded (0.022%)
[8:46] <PoRNo-MoRoZ> recovery 2794/7735910 objects misplaced (0.036%)
[8:46] <PoRNo-MoRoZ> recovery 4/3867677 unfound (0.000%)
[8:47] <motk> 'undersized'
[8:50] <PoRNo-MoRoZ> wtf
[8:50] <PoRNo-MoRoZ> 3 pg undersized that is stuck
[8:51] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[8:52] <PoRNo-MoRoZ> sec looks like they was skipped on my import
[8:52] <PoRNo-MoRoZ> or export
[8:53] * haomaiwang (~haomaiwan@2600:1004:b069:6936:6872:fbb8:9a49:ed6c) has joined #ceph
[8:54] * yatin (~yatin@125.99.244.76) has joined #ceph
[8:55] * yk (~yatin@216.207.42.140) has joined #ceph
[8:59] * yatin (~yatin@125.99.244.76) Quit (Read error: Connection reset by peer)
[8:59] * yatin (~yatin@125.99.244.76) has joined #ceph
[9:00] * yk (~yatin@216.207.42.140) Quit (Read error: Connection reset by peer)
[9:01] * haomaiwang (~haomaiwan@2600:1004:b069:6936:6872:fbb8:9a49:ed6c) Quit (Ping timeout: 480 seconds)
[9:04] <PoRNo-MoRoZ> yep
[9:04] <PoRNo-MoRoZ> i skipped 3 pgs
[9:05] * IvanJobs (~hardes@103.50.11.146) has joined #ceph
[9:05] * dgurtner (~dgurtner@178.197.235.112) Quit (Ping timeout: 480 seconds)
[9:06] <IvanJobs> Anyone has any idea about how to check PG logs in ceph?
[9:06] <PoRNo-MoRoZ> cat /var/lib/ceph/osd.N.log ?
[9:06] <PoRNo-MoRoZ> cat /var/log/ceph/osd.N.log ?
[9:06] <PoRNo-MoRoZ> /var/log, sorry )
[9:08] <PoRNo-MoRoZ> health HEALTH_OK
[9:08] <PoRNo-MoRoZ> BOOM
[9:08] <PoRNo-MoRoZ> THANKS !
[9:08] <PoRNo-MoRoZ> I LOVE YOU GUYS :D
[9:10] * analbeard (~shw@31.113.79.173) has joined #ceph
[9:10] * analbeard1 (~shw@support.memset.com) has joined #ceph
[9:11] * vicente_ (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[9:11] * visored (~Azerothia@4MJAAED8Y.tor-irc.dnsbl.oftc.net) Quit ()
[9:13] <IvanJobs> PoRNo-MoRoZ> I think you misunderstood my words, PG logs are not log files, just something like undo log in db field.
[9:13] * derjohn_mobi (~aj@2001:6f8:1337:0:c47c:e72c:5727:639b) has joined #ceph
[9:13] <IvanJobs> where did PG logs store? levelDB? or anything else?
[9:14] * b0e (~aledermue@213.95.25.82) has joined #ceph
[9:17] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) Quit (Ping timeout: 480 seconds)
[9:18] * Titin (~textual@LFbn-1-1560-65.w90-65.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[9:18] * analbeard (~shw@31.113.79.173) Quit (Ping timeout: 480 seconds)
[9:19] * rendar (~I@host37-18-dynamic.13-79-r.retail.telecomitalia.it) has joined #ceph
[9:19] * fsimonce (~simon@host201-70-dynamic.26-79-r.retail.telecomitalia.it) has joined #ceph
[9:20] * huangjun (~kvirc@113.57.168.154) has joined #ceph
[9:23] <PoRNo-MoRoZ> ah
[9:23] <PoRNo-MoRoZ> sorry
[9:23] <PoRNo-MoRoZ> didn't sleep alot :D
[9:24] <PoRNo-MoRoZ> any way to search and delete orphaned PG's and/or objects within osd ?
[9:26] * huangjun|2 (~kvirc@113.57.168.154) Quit (Ping timeout: 480 seconds)
[9:29] * linjan (~linjan@176.195.205.70) has joined #ceph
[9:30] * yatin (~yatin@125.99.244.76) Quit (Remote host closed the connection)
[9:30] * Kupo1 (~t.wilson@23.111.255.162) Quit (Ping timeout: 480 seconds)
[9:33] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) has joined #ceph
[9:34] * wjw-freebsd (~wjw@176.74.240.1) has joined #ceph
[9:40] * Kupo1 (~t.wilson@23.111.255.162) has joined #ceph
[9:41] * x303 (~Lattyware@chomsky.torservers.net) has joined #ceph
[9:41] * allaok (~allaok@machine107.orange-labs.com) has joined #ceph
[9:41] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) Quit (Ping timeout: 480 seconds)
[9:46] * dgurtner (~dgurtner@178.197.231.64) has joined #ceph
[9:48] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[9:49] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[9:51] * via_ (~via@smtp2.matthewvia.info) has joined #ceph
[9:52] * via (~via@smtp2.matthewvia.info) Quit (Ping timeout: 480 seconds)
[9:57] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[9:57] * haomaiwa_ (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[10:02] * Eduardo_ (~Eduardo@189.196.54.77.rev.vodafone.pt) Quit (Quit: Leaving)
[10:02] * efirs (~firs@c-50-185-70-125.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[10:02] * yatin (~yatin@125.99.244.76) has joined #ceph
[10:05] * haomaiwa_ (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[10:05] * evelu (~erwan@37.163.159.80) has joined #ceph
[10:06] * linjan (~linjan@176.195.205.70) Quit (Ping timeout: 480 seconds)
[10:07] * haomaiwang (~haomaiwan@2600:1004:b069:6936:15ea:6fbf:b804:4bb4) has joined #ceph
[10:07] * allaok (~allaok@machine107.orange-labs.com) Quit (Quit: Leaving.)
[10:08] * allaok (~allaok@machine107.orange-labs.com) has joined #ceph
[10:08] * kefu_ is now known as kefu
[10:10] * EinstCra_ (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[10:11] * x303 (~Lattyware@6AGAAA7FF.tor-irc.dnsbl.oftc.net) Quit ()
[10:11] * zviratko (~Vale@tsn109-201-154-148.dyn.nltelcom.net) has joined #ceph
[10:12] * loth (~Kupo@ip68-3-186-201.ph.ph.cox.net) has joined #ceph
[10:12] * loth (~Kupo@ip68-3-186-201.ph.ph.cox.net) has left #ceph
[10:12] * flisky (~Thunderbi@36.110.40.23) Quit (Ping timeout: 480 seconds)
[10:13] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[10:13] * mnathani2_ (~mnathani_@192-0-149-228.cpe.teksavvy.com) has joined #ceph
[10:15] * flisky (~Thunderbi@36.110.40.25) has joined #ceph
[10:15] * linjan (~linjan@176.195.205.70) has joined #ceph
[10:15] * haomaiwang (~haomaiwan@2600:1004:b069:6936:15ea:6fbf:b804:4bb4) Quit (Ping timeout: 480 seconds)
[10:17] * mnathani (~mnathani_@192-0-149-228.cpe.teksavvy.com) Quit (Ping timeout: 480 seconds)
[10:19] * loth1 (~Kupo@85.17.130.33) Quit (Ping timeout: 480 seconds)
[10:21] * wjw-freebsd (~wjw@176.74.240.1) Quit (Quit: Nettalk6 - www.ntalk.de)
[10:22] * wjw-freebsd (~wjw@176.74.240.1) has joined #ceph
[10:22] * yatin (~yatin@125.99.244.76) Quit (Remote host closed the connection)
[10:25] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) has joined #ceph
[10:26] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[10:28] <titzer> hi cephers
[10:28] * _m4r3k (~oftc-webi@193.179.215.99) has joined #ceph
[10:28] <_m4r3k> Hello
[10:28] <titzer> hi
[10:28] <titzer> is writeback throttle still used?
[10:29] <_m4r3k> I was wondering if anybody considered running CEPH OSD on preemptible VMs
[10:30] * yatin (~yatin@125.99.244.76) has joined #ceph
[10:31] * wjw-freebsd (~wjw@176.74.240.1) Quit (Quit: Nettalk6 - www.ntalk.de)
[10:32] * wjw-freebsd (~wjw@176.74.240.1) has joined #ceph
[10:34] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[10:35] * daiver (~daiver@cpe-98-26-71-226.nc.res.rr.com) has joined #ceph
[10:38] * oniane (~oniane@etno.u-strasbg.fr) Quit (Quit: leaving)
[10:41] * zviratko (~Vale@tsn109-201-154-148.dyn.nltelcom.net) Quit ()
[10:43] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[10:43] * daiver (~daiver@cpe-98-26-71-226.nc.res.rr.com) Quit (Ping timeout: 480 seconds)
[10:46] * djidis__ (~Salamande@94.102.49.175) has joined #ceph
[10:48] * Lea (~LeaChim@host86-176-19-208.range86-176.btcentralplus.com) has joined #ceph
[10:52] * pabluk__ is now known as pabluk_
[10:52] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) Quit (Ping timeout: 480 seconds)
[10:55] * haomaiwang (~haomaiwan@2600:1004:b069:6936:40f9:fba8:e7e0:c17d) has joined #ceph
[10:56] * haomaiwa_ (~haomaiwan@12.222.128.194) has joined #ceph
[11:00] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:00] * haomaiw__ (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[11:02] * zhaochao_ (~zhaochao@124.202.191.137) has joined #ceph
[11:03] * haomaiwang (~haomaiwan@2600:1004:b069:6936:40f9:fba8:e7e0:c17d) Quit (Ping timeout: 480 seconds)
[11:03] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[11:05] * haomaiwa_ (~haomaiwan@12.222.128.194) Quit (Ping timeout: 480 seconds)
[11:07] * haomaiw__ (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Remote host closed the connection)
[11:08] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[11:08] * zhaochao (~zhaochao@125.39.112.5) Quit (Ping timeout: 480 seconds)
[11:08] <joelio> what's the minimal kernel required for 400000000000000 ?
[11:08] <joelio> [ 523.852466] libceph: mon0 192.168.123.1:6789 feature set mismatch, my 107b84a842aca < server's 40107b84a842aca, missing 400000000000000
[11:09] <joelio> Linux rp-node-01 4.4.0-18-generic #34~14.04.1-Ubuntu SMP Thu Apr 7 18:31:54 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[11:10] <joelio> or am I 'doing it wrong'
[11:10] * jowilkin (~jowilkin@c-98-207-136-41.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[11:10] <joelio> updated to jewel last night, got warning about tunables (was seemingly ok on interfernalis with defaults)
[11:10] <joelio> set to optimal and no now cephfs mount
[11:11] <joelio> could drop it to firefly afaik, but wonder which kernel is needed there?
[11:13] <frickler> joelio: 4.5 according to http://docs.ceph.com/docs/master/rados/operations/crush-map/#which-client-versions-support-crush-tunables5
[11:13] <joelio> ok, but isn't jewel in LTS ubuntu
[11:13] <joelio> and that is 4.4
[11:14] <frickler> joelio: yes, so don't set chooseleaf_stable if you need kernel based RBD
[11:15] * yatin (~yatin@125.99.244.76) Quit (Remote host closed the connection)
[11:16] * djidis__ (~Salamande@06SAABKO0.tor-irc.dnsbl.oftc.net) Quit ()
[11:16] * Lunk2 (~skrblr@185.100.85.192) has joined #ceph
[11:16] <joelio> frickler: that a ceph.conf thing or something I can iject?
[11:16] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[11:16] <joelio> ah got it
[11:16] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) Quit (Ping timeout: 480 seconds)
[11:20] * allaok (~allaok@machine107.orange-labs.com) has left #ceph
[11:20] * yanzheng (~zhyan@125.70.21.113) Quit (Quit: This computer has gone to sleep)
[11:22] * dgurtner_ (~dgurtner@178.197.231.64) has joined #ceph
[11:22] * jowilkin (~jowilkin@c-98-207-136-41.hsd1.ca.comcast.net) has joined #ceph
[11:23] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[11:25] * dgurtner (~dgurtner@178.197.231.64) Quit (Ping timeout: 480 seconds)
[11:26] * _m4r3k (~oftc-webi@193.179.215.99) Quit (Ping timeout: 480 seconds)
[11:28] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) Quit (Quit: Leaving)
[11:31] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[11:36] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) has joined #ceph
[11:38] * haomaiwang (~haomaiwan@2600:1004:b069:6936:38e7:92e2:4bcc:36ce) has joined #ceph
[11:39] * i_m (~ivan.miro@88.206.104.168) has joined #ceph
[11:46] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[11:46] * Lunk2 (~skrblr@6AGAAA7JS.tor-irc.dnsbl.oftc.net) Quit ()
[11:46] * haomaiwang (~haomaiwan@2600:1004:b069:6936:38e7:92e2:4bcc:36ce) Quit (Ping timeout: 480 seconds)
[11:46] * evelu (~erwan@37.163.159.80) Quit (Ping timeout: 480 seconds)
[11:46] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:53] * haomaiwang (~haomaiwan@2600:1004:b069:6936:a848:64e3:2573:e4e4) has joined #ceph
[11:54] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Remote host closed the connection)
[11:54] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[11:55] * TMM (~hp@178-84-46-106.dynamic.upc.nl) Quit (Quit: Ex-Chat)
[11:56] * shylesh__ (~shylesh@45.124.227.2) has joined #ceph
[11:56] * evelu (~erwan@37.160.240.147) has joined #ceph
[11:57] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[11:57] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:58] * karnan (~karnan@121.244.87.117) has joined #ceph
[11:59] * Mika_c (~quassel@122.146.93.152) Quit (Remote host closed the connection)
[12:01] * haomaiwang (~haomaiwan@2600:1004:b069:6936:a848:64e3:2573:e4e4) Quit (Ping timeout: 480 seconds)
[12:02] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[12:05] * toMeloos (~toMeloos@53568B3D.cm-6-7c.dynamic.ziggo.nl) has joined #ceph
[12:07] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[12:09] * huangjun (~kvirc@113.57.168.154) Quit (Ping timeout: 480 seconds)
[12:13] * TMM (~hp@31.161.164.58) has joined #ceph
[12:15] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[12:16] * pepzi (~Shadow386@ori.enn.lu) has joined #ceph
[12:17] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[12:17] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[12:21] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Read error: Connection reset by peer)
[12:22] * Vacuum_ (~Vacuum@i59F7999F.versanet.de) has joined #ceph
[12:22] * davidb1 (~David@MTRLPQ42-1176054809.sdsl.bell.ca) Quit (Ping timeout: 480 seconds)
[12:23] <post-factum> how one could list all mds in cluster (with hostnames)?
[12:24] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[12:24] * natarej (~natarej@2001:8003:483a:a900:132:c9d9:1484:c9a5) has joined #ceph
[12:24] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[12:26] * TMM (~hp@31.161.164.58) Quit (Quit: Ex-Chat)
[12:27] * yatin (~yatin@125.99.244.76) has joined #ceph
[12:28] * rraja (~rraja@121.244.87.117) has joined #ceph
[12:30] * davidb (~David@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[12:32] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[12:33] * dneary (~dneary@96.95.216.225) Quit (Ping timeout: 480 seconds)
[12:35] * yatin (~yatin@125.99.244.76) Quit (Ping timeout: 480 seconds)
[12:36] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) has joined #ceph
[12:36] * lmb (~Lars@74.203.127.5) has joined #ceph
[12:41] * haomaiwang (~haomaiwan@2600:1004:b069:6936:e957:ad1b:d42e:dc45) has joined #ceph
[12:44] * daiver (~daiver@2606:a000:111b:c12b:6197:8cb9:dffd:cebb) Quit (Ping timeout: 480 seconds)
[12:44] * Vacuum_ (~Vacuum@i59F7999F.versanet.de) Quit (Ping timeout: 480 seconds)
[12:45] * haomaiwa_ (~haomaiwan@2600:1004:b069:6936:e9d3:8367:9943:b10f) has joined #ceph
[12:46] * pepzi (~Shadow386@6AGAAA7MP.tor-irc.dnsbl.oftc.net) Quit ()
[12:46] * AotC (~CydeWeys@edwardsnowden2.torservers.net) has joined #ceph
[12:49] * haomaiwang (~haomaiwan@2600:1004:b069:6936:e957:ad1b:d42e:dc45) Quit (Ping timeout: 480 seconds)
[12:53] * haomaiwa_ (~haomaiwan@2600:1004:b069:6936:e9d3:8367:9943:b10f) Quit (Ping timeout: 480 seconds)
[12:54] * IvanJobs (~hardes@103.50.11.146) Quit (Read error: Connection reset by peer)
[12:58] * Vacuum_ (~Vacuum@i59F79B18.versanet.de) has joined #ceph
[12:59] * lmb (~Lars@74.203.127.5) Quit (Ping timeout: 480 seconds)
[13:02] * kasimon (~user@2a02:2450:dd1f::2450) has joined #ceph
[13:03] <kasimon> Hi! I'm having a rought time with systemd after updating my debian jessie test cluster from hammer to jewel.
[13:03] <sep> you just skipped infernalis ?
[13:03] <kasimon> After reboot, no ceph services are started.
[13:04] <kasimon> @sep: yes. according to the upgrade notes, that's okay.
[13:05] <kasimon> @sep: also, there is currently no real data on the cluster, so I went straight away.
[13:05] * toMeloos (~toMeloos@53568B3D.cm-6-7c.dynamic.ziggo.nl) Quit (Quit: Ik ga weg)
[13:06] * flisky (~Thunderbi@36.110.40.25) Quit (Quit: flisky)
[13:06] * b0e (~aledermue@213.95.25.82) Quit (Ping timeout: 480 seconds)
[13:07] <sep> kasimon, did you edit the config to still run as root. or did you change file permissions ?
[13:07] * vicente_ (~~vicente@125-227-238-55.HINET-IP.hinet.net) Quit (Quit: Leaving)
[13:07] <sep> also what does logs say ?
[13:07] * TMM (~hp@185.5.122.2) has joined #ceph
[13:07] <sep> i am interested since i am planning hammer-> infernalis soon :)
[13:09] <kasimon> good point regarding the permissions. I did a chown -R ceph:ceph /var/lib/ceph, but /etc/ceph still belongs to root
[13:10] <sep> but can ceph read it ? some of the keyrings atleast are often restricted
[13:10] * TMM (~hp@185.5.122.2) Quit ()
[13:11] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[13:11] <kasimon> I just chown'd /etc/ceph as well
[13:12] * TMM (~hp@185.5.122.2) has joined #ceph
[13:12] <kasimon> But I believe it's more a systemd thing. 'systemctl | grep ceph' lists neither the mon nor the osds
[13:13] <kasimon> The only unit listed is ceph.target, but when I 'systemctl start' that nothing happens.
[13:14] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[13:16] * zhaochao_ (~zhaochao@124.202.191.137) Quit (Quit: ChatZilla 0.9.92 [Firefox 45.0.2/20160413010457])
[13:16] * AotC (~CydeWeys@7V7AAD6MU.tor-irc.dnsbl.oftc.net) Quit ()
[13:17] <sep> are you sure you got the packages properly installed ? do you have the services files ? i have /lib/systemd/system/ceph-osd@.service ;; /lib/systemd/system/ceph-mon@.service ;; /lib/systemd/system/ceph-disk@.service ;; /lib/systemd/system/ceph-create-keys@.service
[13:17] <sep> also my infernalis lab have root:root on /etc/ceph as well
[13:18] <kasimon> Yes, I have them.
[13:19] <kasimon> I also can start services manually, 'systemctl start ceph-mon@$(hostname -s)' works. But it is not started at boot.
[13:20] <kasimon> What I did is to remove /etc/init.d/ceph to make sure only systemd manages the daemons.
[13:20] <kasimon> (But before removing the init file it didn't work either)
[13:21] <kasimon> I did 'systemcl enable' the mon as well es ceph.target
[13:22] <sep> anything in logs ?
[13:22] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[13:22] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[13:23] <kasimon> For example regarding ceph-mon the first mention in the logs is when I manually start it. It looks as if systemd isn't even trying to start it.
[13:23] <kasimon> As if it wasn't aware the service exists at all.
[13:23] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[13:25] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[13:26] * rotbeard (~redbeard@2a02:908:df18:b980:6267:20ff:feb7:c20) Quit (Ping timeout: 480 seconds)
[13:29] * yatin (~yatin@125.99.244.76) has joined #ceph
[13:31] * yk (~yatin@216.207.42.140) has joined #ceph
[13:34] * rakeshgm (~rakesh@121.244.87.117) has joined #ceph
[13:34] * Hemanth (~hkumar_@121.244.87.117) Quit (Quit: Leaving)
[13:35] * mortn (~mortn@217-215-219-69-no229.tbcn.telia.com) has joined #ceph
[13:36] <mortn> on ubuntu 16.04 the ceph-mon is not starting automatically. anybody knows why?
[13:37] * yk (~yatin@216.207.42.140) Quit (Quit: Leaving...)
[13:37] <kasimon> mortn: I'm having the same problem on debian 8. Could you tell me if /lib/systemd/system/ceph-mon.target exists? It's missing here.
[13:37] * yatin (~yatin@125.99.244.76) Quit (Ping timeout: 480 seconds)
[13:38] <mortn> /lib/systemd/system/ceph-mon.target is not there
[13:38] <mortn> only /lib/systemd/system/ceph.target
[13:39] <mortn> seems like /lib/systemd/system/ceph.target is supposed to do both ceph-mons and ceph-osds
[13:39] <kasimon> mortn: Try to put https://github.com/ceph/ceph/blob/master/systemd/ceph-mon.target into that folder.
[13:40] <mortn> and do a systemctl enable ceph-mon.target?
[13:41] <mortn> kasimon: don't know if i'm supposed to systemctl enable ceph-mon.target?
[13:42] * murmur (~murmur@zeeb.org) Quit (Read error: Connection reset by peer)
[13:42] * murmur (~murmur@zeeb.org) has joined #ceph
[13:43] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[13:43] <kasimon> I'm just now testing if adding these sub-targets (ceph-osd.target is missing too) fixes the problem and if will file a bug report.
[13:43] <kasimon> exactly. Currently rebooting to verify this helps.
[13:43] <kasimon> mortn: I would do it as a workaround until the packages are fixed.
[13:43] <kasimon> After reboot ceph-mon is started now.
[13:43] * overclk_ (~quassel@117.202.100.194) Quit (Ping timeout: 480 seconds)
[13:45] * daiver (~daiver@95.85.8.93) has joined #ceph
[13:45] <mortn> kasimon: yay, that works! thank you!
[13:48] <kasimon> mortn: nice. Now I just have to find out why my osds don't start too.
[13:48] <mortn> my osds wouldn't start because i use another SSD for journal on the spinning disks
[13:49] * fdmanana (~fdmanana@74.203.127.5) has joined #ceph
[13:49] <mortn> had to change /usr/lib/ceph/ceph-osd-prestart.sh and add "chown ceph.ceph $journal" just before "exit 0"
[13:53] <kasimon> mortn: that's not the problem here, I can start the osd manually if do 'systemctl start ceph-disk@/dev/cciss/c0d2p1' before.
[13:56] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[13:59] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[13:59] * dgurtner_ (~dgurtner@178.197.231.64) Quit (Ping timeout: 480 seconds)
[13:59] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Remote host closed the connection)
[13:59] * haomaiwang (~haomaiwan@2600:1004:b069:6936:9c18:200b:aac:501f) has joined #ceph
[14:00] * nigwil (~Oz@li1416-21.members.linode.com) Quit (Ping timeout: 480 seconds)
[14:00] * TMM_ (~hp@185.5.122.2) has joined #ceph
[14:00] <mortn> my units are all named ceph-osd@[osd number]
[14:01] * TMM is now known as Guest1443
[14:01] * TMM_ is now known as TMM
[14:01] * Guest1443 (~hp@185.5.122.2) Quit (Ping timeout: 480 seconds)
[14:01] * daiver (~daiver@95.85.8.93) Quit (Remote host closed the connection)
[14:01] * daiver (~daiver@95.85.8.93) has joined #ceph
[14:06] * dgurtner (~dgurtner@178.197.231.64) has joined #ceph
[14:06] * nigwil (~Oz@li1416-21.members.linode.com) has joined #ceph
[14:08] * wyang (~wyang@116.216.0.53) has joined #ceph
[14:09] <kasimon> mortn: my too. But it seems to me the udev rules do not kick in.
[14:10] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) has joined #ceph
[14:11] * davidb (~David@MTRLPQ42-1176054809.sdsl.bell.ca) Quit (Quit: Leaving.)
[14:16] * Bored (~danielsj@exit1.ipredator.se) has joined #ceph
[14:19] * bene2 (~bene@nat-pool-rdu-t.redhat.com) has joined #ceph
[14:35] * fabioFVZ (~fabiofvz@239.78.186.89.cust.ip.kpnqwest.it) has joined #ceph
[14:35] * fabioFVZ (~fabiofvz@239.78.186.89.cust.ip.kpnqwest.it) Quit ()
[14:36] * dgurtner (~dgurtner@178.197.231.64) Quit (Ping timeout: 480 seconds)
[14:37] * wyang (~wyang@116.216.0.53) Quit (Quit: This computer has gone to sleep)
[14:45] * daiver (~daiver@95.85.8.93) Quit (Remote host closed the connection)
[14:45] * daiver (~daiver@95.85.8.93) has joined #ceph
[14:45] * daiver (~daiver@95.85.8.93) Quit (Remote host closed the connection)
[14:46] * daiver (~daiver@95.85.8.93) has joined #ceph
[14:46] * Bored (~danielsj@4MJAAEEEP.tor-irc.dnsbl.oftc.net) Quit ()
[14:48] * Bartek (~Bartek@78.8.183.168) has joined #ceph
[14:48] * mortn (~mortn@217-215-219-69-no229.tbcn.telia.com) Quit (Quit: Leaving)
[14:50] * dgurtner (~dgurtner@178.197.231.64) has joined #ceph
[14:50] * fdmanana (~fdmanana@74.203.127.5) Quit (Ping timeout: 480 seconds)
[14:51] * Bartek (~Bartek@78.8.183.168) Quit (Remote host closed the connection)
[14:51] * mortn (~mortn@217-215-219-69-no229.tbcn.telia.com) has joined #ceph
[14:56] * nigwil (~Oz@li1416-21.members.linode.com) Quit (Ping timeout: 480 seconds)
[14:58] * haomaiwang (~haomaiwan@2600:1004:b069:6936:9c18:200b:aac:501f) Quit (Remote host closed the connection)
[15:05] * fdmanana (~fdmanana@74.203.127.5) has joined #ceph
[15:05] * jdillaman (~jdillaman@pool-108-18-97-82.washdc.fios.verizon.net) has joined #ceph
[15:07] * wyang (~wyang@116.216.30.50) has joined #ceph
[15:08] * johnavp1989 (~jpetrini@8.39.115.8) has joined #ceph
[15:08] * wyang (~wyang@116.216.30.50) Quit ()
[15:10] * toastyde1th (~toast@pool-71-255-253-39.washdc.fios.verizon.net) has joined #ceph
[15:12] * thomnico (~thomnico@12.237.105.253) has joined #ceph
[15:16] * mhack (~mhack@66-168-117-78.dhcp.oxfr.ma.charter.com) has joined #ceph
[15:16] * DV__ (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[15:16] * daiver (~daiver@95.85.8.93) Quit (Remote host closed the connection)
[15:16] * daiver (~daiver@95.85.8.93) has joined #ceph
[15:17] * toastydeath (~toast@pool-71-255-253-39.washdc.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[15:17] * fdmanana (~fdmanana@74.203.127.5) Quit (Ping timeout: 480 seconds)
[15:17] * thomnico_ (~thomnico@12.237.105.253) has joined #ceph
[15:18] * DV__ (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[15:18] * thomnico (~thomnico@12.237.105.253) Quit (Read error: Connection reset by peer)
[15:20] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[15:23] * vikhyat (~vumrao@121.244.87.116) Quit (Quit: Leaving)
[15:28] * brad_mssw (~brad@66.129.88.50) has joined #ceph
[15:30] * mortn (~mortn@217-215-219-69-no229.tbcn.telia.com) Quit (Remote host closed the connection)
[15:32] * jcsp (~jspray@nat-pool-rdu-u.redhat.com) has joined #ceph
[15:32] * jcsp (~jspray@nat-pool-rdu-u.redhat.com) Quit ()
[15:32] * jcsp (~jspray@nat-pool-rdu-u.redhat.com) has joined #ceph
[15:34] * Drankis (~martin@89.111.13.198) has joined #ceph
[15:34] * Drankis (~martin@89.111.13.198) Quit (Remote host closed the connection)
[15:35] * mortn (~mortn@217-215-219-69-no229.tbcn.telia.com) has joined #ceph
[15:37] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[15:43] * wwdillingham (~LobsterRo@189.149.136.30) has joined #ceph
[15:45] * rakeshgm (~rakesh@121.244.87.117) Quit (Quit: Leaving)
[15:45] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[15:48] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) has joined #ceph
[15:48] * wjw-freebsd (~wjw@176.74.240.1) Quit (Ping timeout: 480 seconds)
[15:48] * post-factum (~post-fact@vulcan.natalenko.name) Quit (Quit: leaving)
[15:49] * hybrid512 (~walid@195.200.189.206) has joined #ceph
[15:51] * post-factum (~post-fact@104.207.131.136) has joined #ceph
[15:51] * wwdillingham (~LobsterRo@189.149.136.30) Quit (Quit: wwdillingham)
[15:51] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[15:51] * post-factum (~post-fact@104.207.131.136) Quit ()
[15:53] * csoukup (~csoukup@2605:a601:9c8:6b00:ec0c:d83f:5c09:d1e9) Quit (Ping timeout: 480 seconds)
[15:54] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[15:54] * post-factum (~post-fact@vulcan.natalenko.name) has joined #ceph
[15:55] * wyang (~wyang@116.216.0.53) has joined #ceph
[15:59] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) Quit (Ping timeout: 480 seconds)
[16:03] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[16:03] * csoukup (~csoukup@159.140.254.105) has joined #ceph
[16:04] * MannerMan (~oscar@user170.217-10-117.netatonce.net) Quit (Remote host closed the connection)
[16:06] * i_m (~ivan.miro@88.206.104.168) Quit (Ping timeout: 480 seconds)
[16:09] * post-factum (~post-fact@vulcan.natalenko.name) Quit (Quit: leaving)
[16:09] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Remote host closed the connection)
[16:10] * post-factum (~post-fact@vulcan.natalenko.name) has joined #ceph
[16:10] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:10] * ibravo (~ibravo@72.83.69.64) has joined #ceph
[16:11] * rraja (~rraja@121.244.87.117) Quit (Remote host closed the connection)
[16:11] * Concubidated (~cube@c-50-173-245-118.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[16:12] * thomnico_ (~thomnico@12.237.105.253) Quit (Ping timeout: 480 seconds)
[16:12] * thomnico (~thomnico@12.237.105.2) has joined #ceph
[16:16] * beaver6675 (~beaver667@101.127.60.122) has joined #ceph
[16:17] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) Quit (Ping timeout: 480 seconds)
[16:17] <beaver6675> Hi Cephers, upgrade to Jewel, couldn't see and radosgw buckets from Hammer
[16:17] <beaver6675> s/and/any/
[16:17] <beaver6675> ..is the metadata incompatible?
[16:17] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[16:18] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Ping timeout: 480 seconds)
[16:18] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:18] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[16:19] <beaver6675> jewel has a bunch of pools default.rgw.XXXX
[16:19] <beaver6675> ...didn't seem to read the exisiting hammer pools .rgw.XXXX
[16:20] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[16:20] * galaxyAbstractor (~loft@5.254.102.185) has joined #ceph
[16:20] * wyang (~wyang@116.216.0.53) Quit (Quit: This computer has gone to sleep)
[16:21] * wyang (~wyang@116.216.30.50) has joined #ceph
[16:22] * haplo37 (~haplo37@107-190-32-70.cpe.teksavvy.com) has joined #ceph
[16:23] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Ping timeout: 480 seconds)
[16:23] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Remote host closed the connection)
[16:24] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:25] * thomnico (~thomnico@12.237.105.2) Quit (Ping timeout: 480 seconds)
[16:25] * wyang (~wyang@116.216.30.50) Quit ()
[16:25] * wyang (~wyang@116.216.30.50) has joined #ceph
[16:26] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[16:27] * evelu (~erwan@37.160.240.147) Quit (Ping timeout: 480 seconds)
[16:29] * lmb (~Lars@50.59.37.123) has joined #ceph
[16:29] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Remote host closed the connection)
[16:29] <evilrob> so looking at sizing num_pg in pools. reading http://ceph.com/planet/how-data-is-stored-in-ceph-cluster/
[16:29] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:30] <evilrob> it lists num_pgs should equal (OSDs*100)/replicas rounded up to next power of 2
[16:31] <evilrob> Is that a pretty sound suggested method to follow?
[16:33] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[16:33] <m0zes> yes, it is sound, but it should also be a % of the expected amount of data per pool.
[16:33] <m0zes> http://ceph.com/pgcalc/
[16:34] * kasimon (~user@2a02:2450:dd1f::2450) has left #ceph
[16:34] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Read error: No route to host)
[16:34] <m0zes> the "add pool" button is broken. so start off with the a bunch of pools, delete/rename any and all you want.
[16:34] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:34] <evilrob> yeah. pgcalc is what I've used in the past
[16:35] <evilrob> ah... but it didn't have (or I didn't see) the "logic behind" section below when I last saw it
[16:36] * karnan (~karnan@121.244.87.117) Quit (Ping timeout: 480 seconds)
[16:37] * dugravot6 (~dugravot6@dn-infra-04.lionnois.site.univ-lorraine.fr) Quit (Quit: Leaving.)
[16:37] <evilrob> I'm training a new guy and making him figure out a lot of this as a learning exercise. I'm trying to remember the tools and guidelines I used a few months ago :)
[16:37] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[16:38] * analbeard1 (~shw@support.memset.com) Quit (Quit: Leaving.)
[16:38] * analbeard (~shw@support.memset.com) has joined #ceph
[16:38] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Remote host closed the connection)
[16:38] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[16:38] * lmb (~Lars@50.59.37.123) Quit (Ping timeout: 480 seconds)
[16:39] * lcurtis_ (~lcurtis@47.19.105.250) has joined #ceph
[16:40] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[16:41] * xarses (~xarses@64.124.158.100) has joined #ceph
[16:42] * askb (~askb@61.3.111.105) has joined #ceph
[16:43] * EinstCra_ (~EinstCraz@101.85.207.66) has joined #ceph
[16:43] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Read error: Connection reset by peer)
[16:44] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:44] * EinstCra_ (~EinstCraz@101.85.207.66) Quit (Read error: Connection reset by peer)
[16:48] * lmb (~Lars@50.59.37.123) has joined #ceph
[16:49] * TMM (~hp@185.5.122.2) Quit (Quit: Ex-Chat)
[16:50] * galaxyAbstractor (~loft@4MJAAEEHF.tor-irc.dnsbl.oftc.net) Quit ()
[16:51] * thomnico (~thomnico@12.237.105.2) has joined #ceph
[16:53] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[16:55] * mattbenjamin (~mbenjamin@aa2.linuxbox.com) has joined #ceph
[16:55] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Read error: Connection reset by peer)
[16:55] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[16:55] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[16:57] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[17:01] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[17:01] * dgurtner (~dgurtner@178.197.231.64) Quit (Ping timeout: 480 seconds)
[17:02] * EinstCra_ (~EinstCraz@101.85.207.66) has joined #ceph
[17:02] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Read error: Connection reset by peer)
[17:04] * squizzi (~squizzi@nat-pool-rdu-t.redhat.com) has joined #ceph
[17:05] * shakamunyi (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) has joined #ceph
[17:06] * infernix (nix@2001:41f0::2) Quit (Quit: ZNC - http://znc.sourceforge.net)
[17:06] * infernix (nix@spirit.infernix.net) has joined #ceph
[17:09] * lmb (~Lars@50.59.37.123) Quit (Ping timeout: 480 seconds)
[17:12] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[17:13] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[17:14] * bene2 (~bene@nat-pool-rdu-t.redhat.com) Quit (Ping timeout: 480 seconds)
[17:15] * fdmanana (~fdmanana@74.203.127.5) has joined #ceph
[17:17] * squizzi (~squizzi@nat-pool-rdu-t.redhat.com) Quit (Ping timeout: 480 seconds)
[17:19] * nisha (~nisha@106.78.62.154) has joined #ceph
[17:19] * loicd (~loicd@211.ip-167-114-243.eu) Quit (Quit: quit)
[17:20] * loicd (~loicd@211.ip-167-114-243.eu) has joined #ceph
[17:21] * csharp (~zapu@politkovskaja.torservers.net) has joined #ceph
[17:21] * wyang (~wyang@116.216.30.50) Quit (Quit: This computer has gone to sleep)
[17:22] * i_m (~ivan.miro@83.149.35.74) has joined #ceph
[17:22] * wyang (~wyang@116.216.0.53) has joined #ceph
[17:22] * wyang (~wyang@116.216.0.53) Quit (Remote host closed the connection)
[17:23] * csharp (~zapu@6AGAAA70J.tor-irc.dnsbl.oftc.net) Quit ()
[17:24] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) Quit (Quit: Leaving)
[17:25] * jordanP (~jordan@204.13-14-84.ripe.coltfrance.com) Quit (Quit: Leaving)
[17:25] * Teddybareman (~Sigma@06SAABK01.tor-irc.dnsbl.oftc.net) has joined #ceph
[17:27] <valeech> Hello. I have a lab setup to try out ceph. I have 3 nodes with 8 cores and 196GB RAM each. I have 6 Western Digital Red 5TB drives in each server along with 2 Samsung Pro 950 512GB NVMe drives. Each server has 2 10G Nics installed. I have a very vanilla ceph config at this point. All 18 OSDs are up and in. I have the 950 NVMe drives partitioned with 3 100G partitions. Each 5TB drive is an OSD with a journal living on the NVMe. I created a si
[17:27] <valeech> pool with a 2,1 replication and 1024 PGs.
[17:28] <valeech> My question: Should I be seeing more than 180MB/s transfers with a rados bench with this configuration?
[17:28] <valeech> I use ???rados bench -p ceph 30 write??? to test
[17:29] * EinstCra_ (~EinstCraz@101.85.207.66) Quit (Remote host closed the connection)
[17:30] * nisha (~nisha@106.78.62.154) Quit (Ping timeout: 480 seconds)
[17:30] * dvanders_ (~dvanders@dvanders-pro.cern.ch) Quit (Ping timeout: 480 seconds)
[17:31] * squizzi (~squizzi@nat-pool-rdu-u.redhat.com) has joined #ceph
[17:31] * askb (~askb@61.3.111.105) Quit (Quit: Leaving)
[17:32] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[17:34] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Remote host closed the connection)
[17:35] * Kupo2 (~t.wilson@23.111.255.162) has joined #ceph
[17:39] * fdmanana (~fdmanana@74.203.127.5) Quit (Ping timeout: 480 seconds)
[17:39] * Kupo1 (~t.wilson@23.111.255.162) Quit (Ping timeout: 480 seconds)
[17:39] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Quit: Leaving.)
[17:40] * ibravo (~ibravo@72.83.69.64) Quit (Quit: This computer has gone to sleep)
[17:42] * ibravo (~ibravo@72.83.69.64) has joined #ceph
[17:42] * ibravo (~ibravo@72.83.69.64) Quit ()
[17:48] * georgem (~Adium@206.108.127.16) has joined #ceph
[17:54] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[17:55] * beaver6675 (~beaver667@101.127.60.122) has left #ceph
[17:55] * Teddybareman (~Sigma@06SAABK01.tor-irc.dnsbl.oftc.net) Quit ()
[17:55] * Aramande_ (~kiasyn@tor-exit1-readme.dfri.se) has joined #ceph
[17:57] * nisha (~nisha@106.76.168.226) has joined #ceph
[17:59] * MentalRay_ (~MentalRay@office-mtl1-nat-146-218-70-69.gtcomm.net) has joined #ceph
[18:00] * kefu is now known as kefu|afk
[18:00] * kfox1111 (bob@leary.csoft.net) has joined #ceph
[18:00] <kfox1111> can you run a jewel radosgw and an infernalis cluster?
[18:02] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[18:02] * MentalRay_ (~MentalRay@office-mtl1-nat-146-218-70-69.gtcomm.net) Quit ()
[18:05] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Remote host closed the connection)
[18:06] * nigwil (~Oz@li1416-21.members.linode.com) has joined #ceph
[18:06] * Skaag (~lunix@65.200.54.234) has joined #ceph
[18:07] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) Quit (Ping timeout: 480 seconds)
[18:08] * Skaag (~lunix@65.200.54.234) Quit ()
[18:12] <evilrob> valeech: I'm running 3 nodes with a similar number of OSDs and am getting 171MB/s so basically the same
[18:13] * squizzi (~squizzi@nat-pool-rdu-u.redhat.com) Quit (Ping timeout: 480 seconds)
[18:13] <evilrob> my cluster with 5 nodes and 150 OSDs is faster
[18:13] * kefu|afk (~kefu@114.92.122.74) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[18:14] <evilrob> but more nodes, more OSDs == more speed so that's expected
[18:14] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Quit: Bye guys! (??????????????????? ?????????)
[18:14] * Skaag (~lunix@65.200.54.234) has joined #ceph
[18:15] * derjohn_mobi (~aj@2001:6f8:1337:0:c47c:e72c:5727:639b) Quit (Ping timeout: 480 seconds)
[18:15] * kefu (~kefu@183.193.162.205) has joined #ceph
[18:16] * EinstCrazy (~EinstCraz@101.85.207.66) has joined #ceph
[18:19] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[18:20] * sudocat (~dibarra@192.185.1.20) has joined #ceph
[18:20] <PoRNo-MoRoZ> how can i know real free space of pools ?
[18:21] * dsl_ (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[18:22] <herrsergio> PoRNo-MoRoZ: good question, I would like to know it too
[18:23] * mnathani2_ is now known as mnathani
[18:23] <valeech> evilrob: It seems like I should be getting better performance as just 1 of the WD drives will perform at 190MB/s and I have 18 of them???
[18:24] * sudocat (~dibarra@192.185.1.20) has left #ceph
[18:24] * squizzi (~squizzi@nat-pool-rdu-t.redhat.com) has joined #ceph
[18:25] * sudocat (~dibarra@192.185.1.20) has joined #ceph
[18:25] * Aramande_ (~kiasyn@6AGAAA714.tor-irc.dnsbl.oftc.net) Quit ()
[18:25] <PoRNo-MoRoZ> no way with infernalis ?
[18:26] * gregsfortytwo (~gregsfort@transit-86-181-132-209.redhat.com) Quit (Ping timeout: 480 seconds)
[18:27] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Ping timeout: 480 seconds)
[18:28] <m0zes> ceph df?
[18:28] * sudocat (~dibarra@192.185.1.20) Quit ()
[18:28] <evilrob> I've also got a cluster with 5 nodes and 150OSDs that's turning 700MB/s write and 1.3GB read
[18:28] * sudocat (~dibarra@192.185.1.20) has joined #ceph
[18:32] * hybrid512 (~walid@195.200.189.206) Quit (Remote host closed the connection)
[18:36] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) has joined #ceph
[18:37] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[18:39] * Concubidated (~cube@c-50-173-245-118.hsd1.ca.comcast.net) has joined #ceph
[18:41] * EinstCrazy (~EinstCraz@101.85.207.66) Quit (Remote host closed the connection)
[18:41] * gregsfortytwo (~gregsfort@transit-86-181-132-209.redhat.com) has joined #ceph
[18:45] <Kdecherf> is there a way to know the disk space used by journal when it's a raw block device?
[18:46] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[18:46] * lxo (~aoliva@lxo.user.oftc.net) Quit ()
[18:47] * branto (~branto@nat-pool-brq-t.redhat.com) Quit (Quit: Leaving.)
[18:50] * bniver (~bniver@71-9-144-29.static.oxfr.ma.charter.com) has joined #ceph
[18:51] * nisha (~nisha@106.76.168.226) Quit (Read error: Connection reset by peer)
[18:52] * kefu_ (~kefu@107.191.53.152) has joined #ceph
[18:53] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[18:54] * post-factum (~post-fact@vulcan.natalenko.name) Quit (Quit: leaving)
[18:55] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Read error: No route to host)
[18:55] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[18:55] * shaunm (~shaunm@74.83.215.100) Quit (Ping timeout: 480 seconds)
[18:55] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[18:55] * qable (~Defaultti@tor.exit.relay.dedicatedpi.com) has joined #ceph
[18:55] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[18:56] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[18:57] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[18:57] * kefu (~kefu@183.193.162.205) Quit (Ping timeout: 480 seconds)
[18:59] * post-factum (~post-fact@vulcan.natalenko.name) has joined #ceph
[19:00] * haplo37 (~haplo37@107-190-32-70.cpe.teksavvy.com) Quit (Ping timeout: 480 seconds)
[19:03] * kefu (~kefu@114.92.122.74) has joined #ceph
[19:04] * kefu_ (~kefu@107.191.53.152) Quit (Read error: Connection reset by peer)
[19:04] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[19:08] * nisha (~nisha@27.97.248.6) has joined #ceph
[19:10] * DG1 (~Adium@inet-hqmc06-o.oracle.com) has joined #ceph
[19:10] * mnathani (~mnathani_@192-0-149-228.cpe.teksavvy.com) Quit (Read error: Connection reset by peer)
[19:11] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) Quit (Read error: Connection reset by peer)
[19:11] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) has joined #ceph
[19:13] * Hazelesque_ (~hazel@phobos.hazelesque.uk) Quit (Remote host closed the connection)
[19:14] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[19:14] * qable (~Defaultti@06SAABK3W.tor-irc.dnsbl.oftc.net) Quit (Remote host closed the connection)
[19:15] * vbellur (~vijay@2601:18f:700:55b0:5e51:4fff:fee8:6a5c) Quit (Ping timeout: 480 seconds)
[19:15] * aiicore_ (~aiicore@s30.linuxpl.com) Quit (Remote host closed the connection)
[19:15] * aiicore (~aiicore@s30.linuxpl.com) has joined #ceph
[19:15] * Superdawg (~Superdawg@ec2-54-243-59-20.compute-1.amazonaws.com) Quit (Remote host closed the connection)
[19:15] * SWAT_ (~swat@cyberdyneinc.xs4all.nl) Quit (Remote host closed the connection)
[19:15] * Superdawg (~Superdawg@ec2-54-243-59-20.compute-1.amazonaws.com) has joined #ceph
[19:15] * Hazelesque (~hazel@phobos.hazelesque.uk) has joined #ceph
[19:16] * zigo (~quassel@182.54.233.6) Quit (Remote host closed the connection)
[19:17] * dec (~dec@223.119.197.104.bc.googleusercontent.com) Quit (Quit: bye)
[19:17] <diq> I'm getting "no space left on device" from a CephFS mount, though it shows 76% used via POSIXC and ceph df numbers match up.
[19:17] <diq> anyone else run into this? Running on infernalis
[19:17] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[19:17] * ffilz (~ffilz@c-76-115-190-27.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[19:18] * aarontc (~aarontc@2001:470:e893::1:1) Quit (Ping timeout: 480 seconds)
[19:18] * joao|afk (~joao@8.184.114.89.rev.vodafone.pt) Quit (Ping timeout: 480 seconds)
[19:18] * wushudoin (~wushudoin@38.140.108.2) Quit (Ping timeout: 480 seconds)
[19:18] * etienneme (~arch@75.ip-167-114-228.eu) Quit (Ping timeout: 480 seconds)
[19:18] * irq0 (~seri@amy.irq0.org) Quit (Ping timeout: 480 seconds)
[19:18] * mattbenjamin (~mbenjamin@aa2.linuxbox.com) Quit (Ping timeout: 480 seconds)
[19:19] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) Quit (Quit: billwebb)
[19:19] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) has joined #ceph
[19:19] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) Quit ()
[19:20] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) has joined #ceph
[19:20] * mnathani (~mnathani_@192-0-149-228.cpe.teksavvy.com) has joined #ceph
[19:20] * bearkitten (~bearkitte@cpe-76-172-86-115.socal.res.rr.com) has joined #ceph
[19:20] * aarontc (~aarontc@2001:470:e893::1:1) has joined #ceph
[19:21] * SWAT (~swat@cyberdyneinc.xs4all.nl) has joined #ceph
[19:21] * zigo (~quassel@gplhost-3-pt.tunnel.tserv18.fra1.ipv6.he.net) has joined #ceph
[19:21] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) Quit (Remote host closed the connection)
[19:21] * joao|afk (~joao@8.184.114.89.rev.vodafone.pt) has joined #ceph
[19:22] * wushudoin (~wushudoin@38.140.108.2) has joined #ceph
[19:22] * ffilz (~ffilz@c-76-115-190-27.hsd1.or.comcast.net) has joined #ceph
[19:22] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[19:23] * mattbenjamin (~mbenjamin@aa2.linuxbox.com) has joined #ceph
[19:23] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Read error: No route to host)
[19:24] * dec (~dec@223.119.197.104.bc.googleusercontent.com) has joined #ceph
[19:24] * brad_mssw (~brad@66.129.88.50) Quit (Quit: Leaving)
[19:26] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[19:27] <kfox1111> can you run a jewel radosgw and an infernalis cluster?
[19:28] * haomaiwang (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Read error: No route to host)
[19:29] * haplo37 (~haplo37@107.190.32.70) has joined #ceph
[19:30] * haomaiwa_ (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[19:32] * vbellur (~vijay@nat-pool-bos-u.redhat.com) has joined #ceph
[19:32] * kefu (~kefu@114.92.122.74) Quit (Max SendQ exceeded)
[19:33] * kefu (~kefu@114.92.122.74) has joined #ceph
[19:38] * haomaiwa_ (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Ping timeout: 480 seconds)
[19:41] * etienneme (~arch@75.ip-167-114-228.eu) has joined #ceph
[19:45] * pabluk_ is now known as pabluk__
[19:48] * dsl_ (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[19:49] * haomaiwa_ (~haomaiwan@180.sub-70-193-56.myvzw.com) has joined #ceph
[19:49] * haomaiwa_ (~haomaiwan@180.sub-70-193-56.myvzw.com) Quit (Read error: Connection reset by peer)
[19:54] * zwu_ (~root@58.135.81.96) Quit (Ping timeout: 480 seconds)
[19:54] * davidzlap (~Adium@2605:e000:1313:8003:8c2f:fea2:3c75:8114) Quit (Ping timeout: 480 seconds)
[19:55] * davidzlap (~Adium@cpe-172-91-154-245.socal.res.rr.com) has joined #ceph
[19:56] * dneary (~dneary@50.254.132.37) has joined #ceph
[20:03] * kefu (~kefu@114.92.122.74) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[20:06] * barra204 (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) has joined #ceph
[20:06] * shakamunyi (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[20:08] * shaunm (~shaunm@cpe-74-131-3-55.kya.res.rr.com) has joined #ceph
[20:10] * vbellur (~vijay@nat-pool-bos-u.redhat.com) Quit (Ping timeout: 480 seconds)
[20:12] * dgurtner (~dgurtner@217-162-119-191.dynamic.hispeed.ch) has joined #ceph
[20:15] <xcezzz> PoRNo-MoRoZ: hey
[20:18] * shylesh__ (~shylesh@45.124.227.2) Quit (Remote host closed the connection)
[20:18] * Kupo2 (~t.wilson@23.111.255.162) has left #ceph
[20:19] * squizzi (~squizzi@nat-pool-rdu-t.redhat.com) Quit (Ping timeout: 480 seconds)
[20:20] * Nats (~natscogs@114.31.195.238) Quit (Read error: Connection reset by peer)
[20:20] * Nats (~natscogs@114.31.195.238) has joined #ceph
[20:21] * thomnico (~thomnico@12.237.105.2) Quit (Ping timeout: 480 seconds)
[20:21] * vbellur (~vijay@nat-pool-bos-t.redhat.com) has joined #ceph
[20:24] * squizzi (~squizzi@nat-pool-rdu-t.redhat.com) has joined #ceph
[20:26] <PoRNo-MoRoZ> xcezzz yo
[20:29] * haomaiwang (~haomaiwan@2600:1004:b069:6936:e821:41f:995:7b51) has joined #ceph
[20:30] * Thayli (~LRWerewol@195.40.181.35) has joined #ceph
[20:31] * reed_ (~reed@75-101-54-18.dsl.static.fusionbroadband.com) Quit (Quit: Ex-Chat)
[20:34] <PoRNo-MoRoZ> increasing size to 3 on hot atm
[20:34] <PoRNo-MoRoZ> :D
[20:35] * haomaiwa_ (~haomaiwan@2600:1004:b069:6936:5507:7b2e:e606:4793) has joined #ceph
[20:37] * haomaiwang (~haomaiwan@2600:1004:b069:6936:e821:41f:995:7b51) Quit (Ping timeout: 480 seconds)
[20:38] <xcezzz> nice
[20:38] <xcezzz> did you resolve your problem with unfounded
[20:39] <xcezzz> was there some crazy solar flares this morning lol??? like all sorts of stuff just decided to go wonky at ~8AM EST??? osds, servers, vms, my car battery, my home computer, all started doing crazy stuff hehe
[20:41] * squizzi (~squizzi@nat-pool-rdu-t.redhat.com) Quit (Ping timeout: 480 seconds)
[20:41] * natarej (~natarej@2001:8003:483a:a900:132:c9d9:1484:c9a5) Quit (Read error: Connection reset by peer)
[20:41] * natarej (~natarej@2001:8003:483a:a900:132:c9d9:1484:c9a5) has joined #ceph
[20:42] * whydidyoustealmynick (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) has joined #ceph
[20:42] * barra204 (~shakamuny@c-67-180-191-38.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[20:42] * saru95 (67334b90@107.161.19.109) has joined #ceph
[20:43] * khyron (~khyron@fixed-190-159-187-190-159-75.iusacell.net) Quit (Quit: The computer fell asleep)
[20:43] * haomaiwa_ (~haomaiwan@2600:1004:b069:6936:5507:7b2e:e606:4793) Quit (Ping timeout: 480 seconds)
[20:44] * dgurtner (~dgurtner@217-162-119-191.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[20:48] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[20:50] * thomnico (~thomnico@12.237.105.2) has joined #ceph
[20:50] * davidzlap (~Adium@cpe-172-91-154-245.socal.res.rr.com) Quit (Quit: Leaving.)
[20:51] <xcezzz> got map eXXXX wrongly marked me down for almost exactly the same number of OSDs in our cluster???
[20:51] * nisha (~nisha@27.97.248.6) Quit (Read error: Connection reset by peer)
[20:54] * davidzlap (~Adium@2605:e000:1313:8003:bc17:6434:487e:488f) has joined #ceph
[20:59] * Thayli (~LRWerewol@6AGAAA8CO.tor-irc.dnsbl.oftc.net) Quit ()
[21:04] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Ping timeout: 480 seconds)
[21:07] * lmb (~Lars@69.38.252.84) has joined #ceph
[21:07] * rendar (~I@host37-18-dynamic.13-79-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[21:10] * rendar (~I@host37-18-dynamic.13-79-r.retail.telecomitalia.it) has joined #ceph
[21:13] * rwheeler (~rwheeler@pool-173-48-195-215.bstnma.fios.verizon.net) has joined #ceph
[21:13] * saru95 (67334b90@107.161.19.109) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[21:20] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[21:21] * Skaag (~lunix@65.200.54.234) Quit (Quit: Leaving.)
[21:21] <PoRNo-MoRoZ> xcezzz YES
[21:21] <xcezzz> sweetness
[21:22] <PoRNo-MoRoZ> i made an script that automatically finds problematic pgs and export it
[21:22] <PoRNo-MoRoZ> later i just massimported it
[21:22] <PoRNo-MoRoZ> and it worked
[21:22] <PoRNo-MoRoZ> i just need ti kick it abit :D
[21:22] <PoRNo-MoRoZ> thankd
[21:22] <PoRNo-MoRoZ> thanks
[21:23] * mortn (~mortn@217-215-219-69-no229.tbcn.telia.com) Quit (Quit: Leaving)
[21:26] * dneary (~dneary@50.254.132.37) Quit (Ping timeout: 480 seconds)
[21:30] * Gibri (~SEBI@06SAABLDF.tor-irc.dnsbl.oftc.net) has joined #ceph
[21:33] * nils__ (~nils@port-19252.pppoe.wtnet.de) Quit (Quit: Ex-Chat)
[21:34] * khyron (~khyron@201.175.38.233) has joined #ceph
[21:39] * TMM_ (~hp@178-84-46-106.dynamic.upc.nl) has joined #ceph
[21:39] * _28_ria (~kvirc@opfr028.ru) Quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/)
[21:41] * derjohn_mobi (~aj@x4db0c2f6.dyn.telefonica.de) has joined #ceph
[21:46] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[21:47] * linuxkidd (~linuxkidd@241.sub-70-210-192.myvzw.com) Quit (Quit: Leaving)
[21:52] * shohn (~shohn@dslb-092-078-051-109.092.078.pools.vodafone-ip.de) Quit (Quit: Leaving.)
[21:57] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[22:02] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) has joined #ceph
[22:04] * bniver (~bniver@71-9-144-29.static.oxfr.ma.charter.com) Quit (Remote host closed the connection)
[22:04] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) Quit (Quit: valeech)
[22:05] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Ping timeout: 480 seconds)
[22:11] * rwheeler (~rwheeler@pool-173-48-195-215.bstnma.fios.verizon.net) Quit (Quit: Leaving)
[22:13] * mattbenjamin (~mbenjamin@aa2.linuxbox.com) Quit (Quit: Leaving.)
[22:14] * linuxkidd (~linuxkidd@241.sub-70-210-192.myvzw.com) has joined #ceph
[22:15] * daiver_ (~daiver@216.85.162.34) has joined #ceph
[22:17] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[22:21] * biGGer (~Jase@65.19.167.132) has joined #ceph
[22:21] * daiver (~daiver@95.85.8.93) Quit (Ping timeout: 480 seconds)
[22:23] * daiver_ (~daiver@216.85.162.34) Quit (Ping timeout: 480 seconds)
[22:23] * vbellur (~vijay@nat-pool-bos-t.redhat.com) Quit (Quit: Leaving.)
[22:26] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Ping timeout: 480 seconds)
[22:28] * georgem (~Adium@206.108.127.16) Quit (Ping timeout: 480 seconds)
[22:29] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[22:30] * lmb (~Lars@69.38.252.84) Quit (Ping timeout: 480 seconds)
[22:36] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) Quit (Quit: billwebb)
[22:38] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) has joined #ceph
[22:42] * technil (~technil@host.cctv.org) has joined #ceph
[22:44] * billwebb (~billwebb@50-203-47-138-static.hfc.comcastbusiness.net) Quit (Quit: billwebb)
[22:45] <technil> hello all, if I am rotating out a host from a cluster, and I have removed the osds and/or mon from that host, there is still an entry for that host as part of the crushmap. (with a weight of 0) can this be deleted from the crushmap?
[22:47] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[22:48] <Gugge-47527> technil: i would expect "ceph crush rm" to be able to do that
[22:48] <Gugge-47527> "ceph osd crush rm" that is
[22:50] * Bartek (~Bartek@dynamic-78-8-169-160.ssp.dialog.net.pl) has joined #ceph
[22:51] * biGGer (~Jase@76GAAERMO.tor-irc.dnsbl.oftc.net) Quit ()
[22:51] * csharp (~Jyron@orion.enn.lu) has joined #ceph
[22:52] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[22:55] <technil> @Gugge-47527 thanks, I had only used that with particular osd.{id} before, it wasn't clear I could remove a "host" with it, I will look deeper.
[22:55] * sigsegv (~sigsegv@188.25.21.170) has joined #ceph
[22:55] * csoukup (~csoukup@159.140.254.105) Quit (Ping timeout: 480 seconds)
[23:00] * haomaiwang (~haomaiwan@105.sub-70-193-32.myvzw.com) has joined #ceph
[23:01] * wak (~oftc-webi@104.132.1.65) has joined #ceph
[23:01] * wak is now known as william
[23:02] * william is now known as wakiii
[23:02] * wakiii is now known as wak_work
[23:04] * sigsegv (~sigsegv@188.25.21.170) has left #ceph
[23:06] * technil (~technil@host.cctv.org) Quit (Quit: Ex-Chat)
[23:08] * haomaiwang (~haomaiwan@105.sub-70-193-32.myvzw.com) Quit (Ping timeout: 480 seconds)
[23:09] <DG1> Has anyone tried https://jclouds.apache.org/guides/openstack/ (jclouds api) with Ceph radosgw?
[23:10] * TMM_ is now known as TMM
[23:11] * _28_ria (~kvirc@opfr028.ru) has joined #ceph
[23:11] <TMM> Is it possible to add a cluster network to an existing cluster?
[23:11] * Gjax (~martin@93-167-84-102-static.dk.customer.tdc.net) Quit (Quit: Leaving)
[23:12] <TMM> And, if I do, do I need to add the ip addresses to ceph.conf for each of my osds?
[23:12] <TMM> I have quite a few and they aren't sequential
[23:12] * _28_ria (~kvirc@opfr028.ru) Quit (Remote host closed the connection)
[23:13] * _28_ria (~kvirc@opfr028.ru) has joined #ceph
[23:13] * _28_ria (~kvirc@opfr028.ru) Quit (Remote host closed the connection)
[23:14] * _28_ria (~kvirc@opfr028.ru) has joined #ceph
[23:14] <The1_> you need to have every daemon down at the same time
[23:15] <The1_> but you should be able to only add the network
[23:16] <The1_> I have this set in my ceoh.conf
[23:16] <The1_> public_network = 192.168.80.0/22
[23:16] <The1_> cluster_network = 192.168.16.0/23
[23:16] <The1_> mon_initial_members = ceph1,ceph2,ceph3
[23:16] <The1_> mon_host = 192.168.82.1,192.168.82.3,192.168.83.5
[23:16] <The1_> ceph.conf even
[23:16] <The1_> and no IPs other than the 3 MONs specified
[23:16] * _28_ria (~kvirc@opfr028.ru) Quit (Remote host closed the connection)
[23:16] * _28_ria (~kvirc@opfr028.ru) has joined #ceph
[23:16] * _28_ria (~kvirc@opfr028.ru) Quit (Remote host closed the connection)
[23:18] * DG1 (~Adium@inet-hqmc06-o.oracle.com) has left #ceph
[23:21] * csharp (~Jyron@76GAAERNQ.tor-irc.dnsbl.oftc.net) Quit ()
[23:21] * jwandborg (~Nephyrin@exit1.ipredator.se) has joined #ceph
[23:29] * haomaiwang (~haomaiwan@2600:1004:b058:b362:89a3:9308:b528:1995) has joined #ceph
[23:31] * diq (~diq@2620:11c:f:2:c23f:d5ff:fe62:112c) Quit (Quit: Leaving)
[23:32] * diq (~diq@2620:11c:f:2:c23f:d5ff:fe62:112c) has joined #ceph
[23:34] * debian112 (~bcolbert@24.126.201.64) Quit (Quit: Leaving.)
[23:35] <TMM> The1_, I have to take down the entire cluster for that?
[23:35] <The1_> TMM: afaik, yes
[23:36] <The1_> try and search the mailinglist
[23:36] * thomnico (~thomnico@12.237.105.2) Quit (Quit: Ex-Chat)
[23:37] <The1_> the question pops up once in a while, and I've never seen any other recommended way
[23:37] * haomaiwang (~haomaiwan@2600:1004:b058:b362:89a3:9308:b528:1995) Quit (Ping timeout: 480 seconds)
[23:38] <TMM> but osds can talk over both the front and the backend networks, right?
[23:38] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) has joined #ceph
[23:38] <TMM> you'd think if the mons are on both networks the osds should wolk
[23:38] <TMM> work*
[23:40] * Gibri (~SEBI@06SAABLDF.tor-irc.dnsbl.oftc.net) Quit (Ping timeout: 480 seconds)
[23:40] <The1_> I'm not sure the MONs and OSDs can handle it..
[23:40] <mnathani> I am new to ceph. Does using it require in depth knowledge of linux / command line / filesystems or does it have a gui front end, perhaps web based?
[23:41] <TMM> mnathani, you will need to get familiar with the cli at least to a certain extent
[23:41] <TMM> mnathani, if you follow all recommended practices you should have little problems, but troubleshooting will require knowledge
[23:41] <mnathani> ok
[23:42] <The1_> if you have no prior knowledge of how to work the shell and administer a *nix machine without a GUI you are in for a hard time..
[23:43] <TMM> mnathani, you need little knowledge to get it running, but again, if you have trouble you will be in deep trouble if you don't have a reasonable knowledge of unix systems
[23:43] <mnathani> does ceph run on top of a certain distro
[23:44] <mnathani> or is it a distro in itself?
[23:44] <The1_> read the docs for recommendations
[23:44] <TMM> mnathani, you need to start here: http://docs.ceph.com/docs/master/start/
[23:44] <TMM> mnathani, try it in a couple of virtual machines so you can easily try again and again, then try to understand what the commands do
[23:48] <TMM> mnathani, in short, you can deploy ceph on a variety of distros. You may want to start just trying to run something simple on a coupe of distros and see which ones you like best. Try perhaps manually installing a simple web application like drupal or wordpress
[23:48] <TMM> mnathani, using the cli, not the guis
[23:50] <mnathani> TMM: right
[23:51] <ben3> you really don't want to use ceph on less than 3 servers
[23:51] * jwandborg (~Nephyrin@76GAAEROP.tor-irc.dnsbl.oftc.net) Quit ()
[23:51] <mnathani> as a cluster
[23:51] <mnathani> ?
[23:51] <TMM> ben3, I'd say no less than 5
[23:51] * ItsCriminalAFK1 (~Aethis@06SAABLI2.tor-irc.dnsbl.oftc.net) has joined #ceph
[23:51] <ben3> and even 3 is questionable for anything more than testing
[23:51] <mnathani> do vms count
[23:51] <TMM> no
[23:51] <mnathani> or 3-5 physical boxes?
[23:51] <TMM> for production you really need to have 5 physical boxes as a minimum
[23:52] <ben3> i tested with 3 physical boxes myself
[23:52] * The1_ whispers that he has a production cluster on 3 nodes holding quite a lot.. ;)
[23:52] <TMM> you can test with vms though, to get familiar with the system
[23:52] <The1_> but I'll be adding to it quite soon
[23:52] <mnathani> is there a recommended minimum storage or drives per box
[23:52] <ben3> The1_: are you using raid within the boxes?
[23:52] <TMM> if you need to actually test workloads you'll need 3 physical boxes
[23:52] <ben3> problem with 3 boxes, is if you're using 3 way replication, you lose any one box and you can't write anything
[23:52] <The1_> ben3: only for OS and journal disks
[23:53] <ben3> why raid journal disks?
[23:53] <ben3> The1_: so are you doing 2 or 3 way minimum replication?
[23:53] <mnathani> I would need a minimum of 10gbit networking, or would a dedicated 1gbit storage network work as well?
[23:53] <The1_> bdecause I don't want a single ssd failure to take down the either the entire node or the OSD it's the journal for
[23:53] <TMM> mnathani, I'd suggest a minimal of 5 systems with a minimum of 5 drives each. That way you can run in a 3/2 setup and you can actually lose some disks and servers without issue
[23:53] * poli (~poli@186.204.210.213) has joined #ceph
[23:54] <ben3> mnathani: just go back to back 10gbe
[23:54] <The1_> ben3: mininum 1 copy
[23:54] <ben3> The1_: sounds unsafe
[23:54] <The1_> I have a full backup of everything
[23:54] * dsl (~dsl@72-48-250-184.dyn.grandenetworks.net) Quit (Remote host closed the connection)
[23:54] <The1_> so for us it's quite safe
[23:54] <TMM> mnathani, 1gbit works, but it is not great. You will have long recovery times if something goes wrong in your cluster, and every gigabit that comes into your networks in writes will cause at least 3x the writes on the backend network.
[23:54] <ben3> The1_: ahh
[23:55] <ben3> 1 gigabit isn't terribly bad for clients of the cluster
[23:55] <ben3> but the cluster itself should be at least 10 gigabit
[23:55] <The1_> ben3: I'm a suspenders, belt, lifewest and random piece of string I found in the roadside kind of guy.. ;)
[23:55] <ben3> i tested wit infiniband and gbe for clients
[23:55] <TMM> mnathani, if your cluster is recovering you will additionally have a ton of traffic on the backend network for the recovery itself. Your cluster will slow to a crawl
[23:56] <ben3> and there was about 2x random peformance boost from infiniband compared to 10gbe
[23:56] <ben3> err compared to gbe even
[23:56] <ben3> if you're recovering with few disks you'll get more load too
[23:57] <ben3> one of the cool things about ceph is when you have a decent sized cluster, you get a single disk failure and it's not backed against one other, but instead many, so it spreads the load
[23:58] <TMM> more disks is certainly better for recovery times
[23:59] <ben3> it is slightly intimidating for smaller setups when the internet suggests it's the normal to have 40gbe etc
[23:59] <ben3> it's the norm even
[23:59] <TMM> I have 30 boxes with 40gbit and 8x1tb ssds in each one :P
[23:59] <The1_> 40Gb is for cross-rack interconnects
[23:59] <TMM> 2x180gbit cross-rack interconnects
[23:59] <ben3> TMM: exactly :)
[23:59] <The1_> haha
[23:59] <TMM> ladieda ;)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.