#ceph IRC Log

Index

IRC Log for 2014-04-05

Timestamps are in GMT/BST.

[0:00] * sz0 (~user@208.72.139.54) Quit (Quit: ERC Version 5.3 (IRC client for Emacs))
[0:02] * sz0 (~user@208.72.139.54) has joined #ceph
[0:02] * nwat (~textual@eduroam-237-164.ucsc.edu) has joined #ceph
[0:05] * scuttlemonkey (~scuttlemo@99-6-62-94.lightspeed.livnmi.sbcglobal.net) has joined #ceph
[0:05] * ChanServ sets mode +o scuttlemonkey
[0:13] <vilobh> dmick : Since you are talking about openstack, in openstack context if we want to create volume/ delete volume/ attach volume /detach volume we don't need kernel block device right (rbd.ko) ? as we can do that using the openstack->libvirt->qemu->librbd->librados way right ?
[0:14] <dmick> right
[0:14] <vilobh> dmick: which part :) first or second ? or both :)
[0:14] <dmick> all of what you said is right
[0:15] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[0:16] <vilobh> dmick: cool but then if we want to take snapshot and clones same path openstack->libvirt->qemu->librbd->librados should be good right ?
[0:16] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[0:16] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) Quit (Ping timeout: 480 seconds)
[0:16] <vilobh> dmick: if we can use openstack->libvirt->qemu->librbd->librados for everything that rbd.ko can do then why do we actually need rbd.ko
[0:17] <dmick> you might not want to use openstack
[0:17] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[0:17] <dmick> or qemu
[0:17] <vilobh> dmick : cool. got it. great thanks!
[0:17] * t0rn (~ssullivan@c-24-11-198-35.hsd1.mi.comcast.net) Quit (Quit: Leaving.)
[0:17] <dmick> do you have a use case in mind, or are you just mapping the capabilities of ceph?
[0:18] <vilobh> dmick: have few use case in mind just was evaluating when i might need rbd.ko and when librbd approach
[0:20] <vilobh> just to reiterate what you said "openstack->libvirt->qemu->librbd->librados can do everything that rbd.ko can do"
[0:20] <vilobh> dmick: ^^
[0:22] <dmick> unless you include "be a kernel block device" in that "everything"
[0:23] <vilobh> dmick: true. great thanks !
[0:25] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Ping timeout: 480 seconds)
[0:27] * BillK (~BillK-OFT@124-148-94-184.dyn.iinet.net.au) has joined #ceph
[0:36] * scuttlemonkey (~scuttlemo@99-6-62-94.lightspeed.livnmi.sbcglobal.net) Quit (Ping timeout: 480 seconds)
[0:37] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[0:39] * ghartz (~ghartz@ip-68.net-80-236-84.joinville.rev.numericable.fr) Quit (Quit: Quitte)
[0:48] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[0:51] * vilobh (~vilobhmm@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Quit: vilobh)
[0:57] * jeff-YF (~jeffyf@67.23.117.122) Quit (Quit: jeff-YF)
[1:01] * vilobh (~vilobhmm@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[1:07] * markbby (~Adium@168.94.245.1) Quit (Remote host closed the connection)
[1:08] * japuzzo (~japuzzo@ool-4570886e.dyn.optonline.net) Quit (Quit: Leaving)
[1:12] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has joined #ceph
[1:14] <lightspeed> I'm about to rebuild my small 3-node cluster from scratch, and am wondering what the latest advice is regarding OSD design
[1:14] <lightspeed> Specifically, I have 3 hosts with 1 OSD each (1TB Samsung 840 Evo SSDs in each case). The same hosts will also each run a mon daemon, and they each have 1 separate, smaller/cheaper/older SSD which houses their root filesystem
[1:15] <lightspeed> The main reason I'm asking for advice here is because I've just had to give up on my last attempt (which started on v0.48 and ended on v0.72), in which all 3 OSD drives died within a couple of months (and I was not proactive enough to deal with the failures individually before the next failure). I didn't lose any important data as I was still only using the environment for testing purposes. But I'd like to avoid making the same mist
[1:15] <lightspeed> akes again :)
[1:16] <lightspeed> I've provided details of my set up here, if anyone is willing to read it and share their advice: http://pastebin.com/RGUsHcYX
[1:16] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) Quit (Quit: Leaving.)
[1:23] * c74d3 (~c74d3a4eb@2002:4404:712c:0:76de:2bff:fed4:2766) Quit (Remote host closed the connection)
[1:25] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[1:26] * leseb (~leseb@185.21.174.206) Quit (Killed (NickServ (Too many failed password attempts.)))
[1:27] * leseb (~leseb@185.21.174.206) has joined #ceph
[1:31] * alfredodeza (~alfredode@198.206.133.89) has joined #ceph
[1:33] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) Quit (Remote host closed the connection)
[1:33] * sjustwork (~sam@2607:f298:a:607:2590:a8c2:b9bc:4115) Quit (Quit: Leaving.)
[1:35] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) has joined #ceph
[1:39] * rmoe (~quassel@12.164.168.117) Quit (Ping timeout: 480 seconds)
[1:44] * ircolle (~Adium@2601:1:8380:2d9:68e9:6778:af47:22ad) Quit (Quit: Leaving.)
[1:47] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[1:49] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[1:51] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) has joined #ceph
[1:51] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[1:53] <aarontc> lightspeed: you can probably RMA the OCZ drives, at least... but I wouldn't use them as the journals in any case
[1:55] <lightspeed> yeah I'll RMA them if I'm not too late to do so - they were purchased as "refurbished" models, at a nice discount, but with a correspondingly lower warranty period
[1:55] <lightspeed> I can't remember if the warranty was 6 months or 12, but I bought them about 7 months ago
[1:56] * alram (~alram@38.122.20.226) Quit (Quit: leaving)
[1:57] <lightspeed> does that mean you'd also suggest I don't put the journals on the Samsung SSDs?
[1:57] <aarontc> other than that I don't see any issues with your setup...
[1:57] <Sysadmin88> if you have SSDs, why have another SSD for journal?
[1:57] <aarontc> I have no idea. I know there have been lots of examples of OCZ drives wearing out quite early
[1:57] <aarontc> no clue if Samsungs are better, but I haven't heard of as many problems
[1:58] <Sysadmin88> if you want reliability, get enterprise drives
[1:58] <lightspeed> aarontc: cool, thanks for the sanity check
[1:59] <Sysadmin88> do you really need the iops from the SSDs?
[1:59] <lightspeed> yeah I realize enterprise drives would be preferable, but I can't really afford them :)
[1:59] * nwat (~textual@eduroam-237-164.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[1:59] <Sysadmin88> would a few 3TB HDDs do performance wise?
[2:00] <lightspeed> one restriction I haven't mentioned is that these are 1U rackmount servers that can only accommodate 2.5 inch SATA drives
[2:00] * tristanz (~tristanza@209.49.228.209) has joined #ceph
[2:00] <lightspeed> that means if I go for spinning disks I'm limited in choices
[2:01] <Sysadmin88> did you build these specifically for ceph?
[2:01] <lightspeed> Velociraptors pretty much equal SSDs in terms of price per GB
[2:01] <nhm_> lightspeed: I think OCZ went bankrupt. :/
[2:01] <Sysadmin88> velociraptors have 5 year warranty
[2:02] <nhm_> lightspeed: if you use SSDs, journals on the same disk are probably fine. Crucial M500s seem to be a popular choice since they have a capacitor, but I've never tested them myself.
[2:02] <lightspeed> not specifically for ceph (I hadn't quite figured out what I wanted to do with them when I got the first servers)
[2:02] <nhm_> Intel DC S3700s are great but pricey.
[2:02] <lightspeed> *first got the servers
[2:04] <Sysadmin88> cases with 4 3.5" drives in a 1U box are relatively cheap
[2:04] * BillK (~BillK-OFT@124-148-94-184.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[2:04] <lightspeed> ok, if there are no inherent issues with journal + OSD on a single SSD, then I'll stick with doing that
[2:05] <Sysadmin88> ssd has more IOPS and speed than a HDD... what is your networking like?
[2:06] <lightspeed> I use 8 Gbps infiniband for the inter-host network, and 2 Gbps etherchannel out to the rest of the network
[2:07] <Sysadmin88> and you only have 3 OSDs total?
[2:07] * Serbitar hugs infiniband
[2:07] <lightspeed> basically I bought these boxes to replace 3 older ones (6 years old or so) that were struggling, with the intention of doing virtualization and "something different" for storage
[2:07] <lightspeed> yeah just 3 OSDs
[2:07] <Sysadmin88> why not augment the old ones... so you have 6
[2:07] * tristanz (~tristanza@209.49.228.209) Quit (Quit: tristanz)
[2:08] <lightspeed> well, I was also keen to reduce my power consumption - the new servers draw about 50% of the power of the old ones :)
[2:08] <Sysadmin88> just remember with so few OSDs a failure is over 15% of your storage
[2:09] <lightspeed> yeah I've come to appreciate that the hard way
[2:09] <Sysadmin88> currently a failure is 33%... and it takes a while to replicate lots of data. if you have a failure your performance may go right down or stop
[2:09] * dis (~dis@109.110.67.80) Quit (Ping timeout: 480 seconds)
[2:10] <Sysadmin88> i would like one of the 24 bay 4U cases... but i would struggle to afford the raid card etc to successfully power it
[2:11] <lightspeed> I can potentially add more drives into these boxes (they can take 4x 2.5 inch drives each), but don't really want more than 3 hosts in total as this is a home setup
[2:12] <janos> you are definitely going to want to add more drives
[2:12] <Sysadmin88> i have a synology at home. works good enough
[2:12] * elyograg (~oftc-webi@client175.mainstreamdata.com) Quit (Quit: Page closed)
[2:13] <janos> i'm eyeballing this when they come out - for a small cluster as monitors
[2:13] <janos> http://www.minnowboard.org/meet-minnowboard-max/
[2:13] <janos> small cluster, low power. drop 3 of these in
[2:13] <Serbitar> oh wow
[2:13] <Serbitar> 1Ge?
[2:14] <Serbitar> nice
[2:14] <janos> yeah, the bigger model is dual core and more ram. i think for a small cluster they will do
[2:14] <lightspeed> that looks nice, certainly addresses my concerns about power :)
[2:14] <Serbitar> and a proper dc power connector too by the looks of it
[2:14] <janos> yep
[2:14] <Sysadmin88> might be better with E-350 or some of the other itx boards which can take 4 HDDs and lots more ram
[2:14] <Serbitar> whats the display interface like?
[2:15] <janos> hdmi
[2:15] <janos> hd4000 i think onboard
[2:15] <janos> yeah i've messed with some e-450's for osd hosts
[2:16] <Serbitar> has anyone tried ceph osd on arm?
[2:17] <janos> not me
[2:17] * dis (~dis@109.110.66.53) has joined #ceph
[2:18] <Serbitar> hm, so the cubieboard that I have would be cool if it had 1Ge
[2:18] <Sysadmin88> you got an arm with multiple satas and gig ethernet?
[2:19] <Serbitar> my ponderance would be to do one disk per osd, if the cost and power was right
[2:19] * mcms (~mcms@46.224.152.64) has joined #ceph
[2:20] * mcms (~mcms@46.224.152.64) has left #ceph
[2:20] <Serbitar> not space efficient though i suppose
[2:20] * diegows_ (~diegows@190.190.5.238) Quit (Ping timeout: 480 seconds)
[2:21] <nhm_> janos: I bought one of these, but haven't tested it yet: http://www.amazon.com/ECS-Elitegroup-Motherboard-NM70-I2-1-0/dp/B00G237CYQ/ref=sr_1_2?ie=UTF8&qid=1396657244&sr=8-2&keywords=ecs+nm70
[2:21] <Sysadmin88> ITX fits 4x3.5" drives and many ITX boards have 4 SATAs. perfect fit :)
[2:21] <Sysadmin88> how were your 3-450 tests?
[2:21] <Sysadmin88> E-450
[2:21] <janos> nhm_, i've been eyeballing those min-itx's
[2:21] <nhm_> janos: I got it back when it was $66
[2:22] <Sysadmin88> 1U fits 4x3.5" drives i meant
[2:22] <janos> with 3 disks, the e-450's don't even break a sweat really
[2:22] <janos> i dind't test past that, but they defintiely seemed like they would take it fine
[2:22] <nhm_> janos: I decided I wanted the 17w ivy-bridge celeron instead
[2:22] <janos> ooh that sounds solid
[2:23] <Sysadmin88> i've been eyeing one of those as a small virtualization server
[2:23] <janos> i love the progression of power, reduction in saize and wattage going on
[2:23] <nhm_> janos: 1.8GHz dual core 17w celeron soldered right on to the board
[2:23] <janos> *size
[2:23] <nhm_> even supports SSE4.2 so you get hardware CRC32
[2:24] <janos> wow
[2:24] <janos> i love throwing a kill-a-watt on machines to see the low wattage
[2:24] <nhm_> Only downside is it's 3 3Gb/s and 1 6Gb/s SATA
[2:25] <nhm_> Have to wait for the haswell refresh
[2:25] <janos> hrmm, i think i have a e-450 right next to me with 3 osd disks pulling maybe 100w when it's under duress
[2:25] <janos> iirc
[2:26] <janos> currently doiing 67w
[2:26] * xarses (~andreww@12.164.168.117) Quit (Ping timeout: 480 seconds)
[2:26] <Sysadmin88> janos 9tb or 12tb total?
[2:26] <janos> it's still hogging up my kill-a-watt
[2:26] <janos> i wish. i just tossed in some old 1TB's ;(
[2:26] <janos> i need to upgrade them
[2:27] <Sysadmin88> nhm_ the gigabit ethernet is limiting before the SATA most probably
[2:27] <nhm_> Sysadmin88: there's a PCIE slot for extra network
[2:27] <janos> my personal fun money has been busy with my own game rig and my kids. so other toying is suffering
[2:27] <nhm_> Sysadmin88: was thinking of throwing a 10GbE in and seeing how hard I could push it.
[2:27] <Serbitar> then your network card ends up costing more than the mobo :P
[2:28] <janos> i REALLY wish 10GBe would come down to more consumer-friendly/hobby prices
[2:28] <Serbitar> yeah, i have dual bonded 1g to my file server
[2:28] <janos> yeah
[2:28] <Serbitar> managed to get some 2 port 1g pcie cards for 30quid each
[2:28] <nhm_> Serbitar: SDR IB cards are dirt cheap on ebay. ;)
[2:29] <Serbitar> nhm_: oh, really. and the cables too?
[2:29] <janos> i have to balance out my distractions!
[2:29] <nhm_> Serbitar: CX4 or SFP+
[2:29] <Serbitar> we use qdr at work, i suspect that in 18months when that kit is decomissioned I will end up running qdr at home
[2:29] <nhm_> SFP+ isn't insanely bad these days for short runs
[2:30] <janos> have a good evening - it's beer+movie time at chez janos
[2:31] <nhm_> here's a DDR (20GBPS) dual port CX4 card for $39: http://www.ebay.com/itm/Mellanox-InfiniHost-III-Ex-MHGA28-XTC-Infiniband-HBA-20GBPS-Dual-port-Adapter-/151270531655?pt=COMP_EN_Servers&hash=item23386d2247
[2:32] * Karcaw_ (~evan@96-41-200-66.dhcp.elbg.wa.charter.com) has joined #ceph
[2:32] * Karcaw (~evan@96-41-200-66.dhcp.elbg.wa.charter.com) Quit (Read error: Connection reset by peer)
[2:33] <nhm_> ooh, you can get connectx-2 10GbE cards on ebay for $69 now.
[2:35] <Serbitar> spotted a qlogic qdr card for gbp50
[2:37] * mcms (~mcms@46.224.152.64) has joined #ceph
[2:38] * sjusthm (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) has joined #ceph
[2:39] <Sysadmin88> but how much is a 10GbE switch on ebay?
[2:39] <lightspeed> those of you considering IB - how would you do the cabling? with 3 hosts each with a dual-port IB card I've been able to have a direct cable between each pair of hosts, but with any more hosts than that it would presumably need an IB switch as well...
[2:39] <Serbitar> you would probably want two switches
[2:39] <lightspeed> yeah exactly :)
[2:40] <Serbitar> as you can bond the two switches together with several cables
[2:40] <Serbitar> and then you will hve fault tolerance
[2:40] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[2:40] <Serbitar> make sure you pick one switch to be the sm
[2:41] <nhm_> lightspeed: I've heard some folks take a system and throw a bunch of dual port IB cards in it and make their own switch. ;)
[2:41] <Serbitar> heh
[2:41] <nhm_> lightspeed: but SDR/DDR IB switches aren't too insane
[2:42] <Serbitar> the downside is that your pc probably has less bandwidth than the switch internal backplane
[2:42] <nhm_> Serbitar: indeed
[2:42] <nhm_> probably fine for SDR, but definitely not QDR
[2:46] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[2:47] * glzhao (~glzhao@123.125.124.17) Quit (Remote host closed the connection)
[2:47] * glzhao (~glzhao@123.125.124.17) has joined #ceph
[2:53] * glzhao (~glzhao@123.125.124.17) Quit (Remote host closed the connection)
[2:54] * glzhao (~glzhao@123.125.124.17) has joined #ceph
[2:54] <lightspeed> any opinions on BTRFS vs XFS on an SSD OSD?
[2:55] * tristanz (~tristanza@209.49.228.209) has joined #ceph
[2:59] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[3:00] <ponyofdeath> hi, i am wondering if i can remove the default pools that are created, like data, rbd ?
[3:00] * sputnik1_ (~sputnik13@207.8.121.241) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[3:03] * tristanz (~tristanza@209.49.228.209) Quit (Ping timeout: 480 seconds)
[3:03] * gregmark (~Adium@68.87.42.115) Quit (Quit: Leaving.)
[3:04] * mcms (~mcms@46.224.152.64) Quit (Ping timeout: 480 seconds)
[3:05] * glzhao (~glzhao@123.125.124.17) Quit (Remote host closed the connection)
[3:06] * glzhao (~glzhao@123.125.124.17) has joined #ceph
[3:07] <Kioob> lightspeed: I think btrfs is not yet ready for production. So : XFS.
[3:09] <nhm_> lightspeed: BTRFS will fragment if you do small writes with RBD
[3:09] * JeffK (~Jeff@38.99.52.10) Quit (Quit: JeffK)
[3:12] <ponyofdeath> nhm_: will the autodefrag help that
[3:13] * zerick (~eocrospom@190.187.21.53) Quit (Remote host closed the connection)
[3:14] <lightspeed> ok thanks, I'll give XFS a go this time round then
[3:15] * Pedras1 (~Adium@216.207.42.132) Quit (Ping timeout: 480 seconds)
[3:19] <nhm_> ponyofdeath: it will, but apparently it can go OOM
[3:19] <nhm_> ponyofdeath: especially with snapshots
[3:20] <ponyofdeath> oom?
[3:20] <ponyofdeath> memory
[3:21] * angdraug (~angdraug@12.164.168.117) Quit (Quit: Leaving)
[3:24] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[3:33] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[3:38] * BillK (~BillK-OFT@106-69-172-170.dyn.iinet.net.au) has joined #ceph
[3:44] * vilobh (~vilobhmm@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Quit: vilobh)
[3:49] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has joined #ceph
[3:51] * sjusthm (~sam@24-205-43-60.dhcp.gldl.ca.charter.com) Quit (Quit: Leaving.)
[3:59] * wrale (~wrale@cpe-107-9-20-3.woh.res.rr.com) has joined #ceph
[4:06] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[4:06] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[4:13] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[4:27] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[4:35] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[4:41] * c74d3 (~c74d3a4eb@2002:4404:712c:0:76de:2bff:fed4:2766) has joined #ceph
[4:43] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[4:45] * dmick (~dmick@2607:f298:a:607:39ed:84cc:5a53:e057) Quit (Quit: Leaving.)
[4:52] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[4:52] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[4:57] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[5:00] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has left #ceph
[5:06] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) has joined #ceph
[5:08] * Vacum (~vovo@i59F79B3D.versanet.de) has joined #ceph
[5:10] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[5:10] * Vacum_ (~vovo@i59F7AFCE.versanet.de) Quit (Read error: Operation timed out)
[5:14] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) Quit (Quit: Leaving.)
[5:28] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[5:51] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[6:01] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[6:04] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has left #ceph
[6:13] * Cube (~Cube@66-87-65-82.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[6:33] * scuttlemonkey (~scuttlemo@c-107-5-193-244.hsd1.mi.comcast.net) has joined #ceph
[6:33] * ChanServ sets mode +o scuttlemonkey
[6:42] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[6:45] * Cube (~Cube@66-87-65-82.pools.spcsdns.net) has joined #ceph
[7:00] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[7:01] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[7:01] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) Quit ()
[7:02] * JoeGruher (~JoeGruher@jfdmzpr05-ext.jf.intel.com) Quit (Remote host closed the connection)
[7:09] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[7:13] * zack_dolby (~textual@e0109-114-22-65-255.uqwimax.jp) has joined #ceph
[7:17] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[7:17] * sz0 (~user@208.72.139.54) Quit (Ping timeout: 480 seconds)
[7:25] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[7:25] <classicsnail> out of memory
[7:26] <classicsnail> I've not had problems using normal mounts with btrfs over the years, less corruption and bad events than even with ext4 with paranoia mount options
[7:26] <classicsnail> but I've had some weirdness I couldn't quite explain with ceph and btrfs osds
[7:27] <classicsnail> and I haven't had the same weirdness with xfs osds on the same systems
[7:31] * leseb (~leseb@185.21.174.206) Quit (Killed (NickServ (Too many failed password attempts.)))
[7:33] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[7:33] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[7:34] * leseb (~leseb@185.21.174.206) has joined #ceph
[7:37] * zack_dolby (~textual@e0109-114-22-65-255.uqwimax.jp) Quit (Ping timeout: 480 seconds)
[7:41] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[7:41] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[7:48] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[7:52] * Boltsky (~textual@office.deviantart.net) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[7:57] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[8:00] * wrale (~wrale@cpe-107-9-20-3.woh.res.rr.com) Quit (Ping timeout: 480 seconds)
[8:04] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[8:07] * Boltsky (~textual@cpe-198-72-138-106.socal.res.rr.com) has joined #ceph
[8:13] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[8:16] * Gugge-47527 (gugge@kriminel.dk) Quit (Read error: Connection reset by peer)
[8:17] * Gugge-47527 (gugge@kriminel.dk) has joined #ceph
[8:20] * sz0 (~user@208.72.139.54) has joined #ceph
[8:21] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[8:22] * Cube (~Cube@66-87-65-82.pools.spcsdns.net) Quit (Quit: Leaving.)
[8:24] * madkiss (~madkiss@2001:6f8:12c3:f00f:b475:434e:e061:a982) Quit (Ping timeout: 480 seconds)
[8:25] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[8:28] * sz0 (~user@208.72.139.54) Quit (Ping timeout: 480 seconds)
[8:29] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[8:32] * JCL (~JCL@2601:9:5980:39b:c526:e507:e4c4:579b) has joined #ceph
[8:32] * JCL (~JCL@2601:9:5980:39b:c526:e507:e4c4:579b) Quit (Remote host closed the connection)
[8:34] * JCL (~JCL@2601:9:5980:39b:c526:e507:e4c4:579b) has joined #ceph
[8:34] * JCL (~JCL@2601:9:5980:39b:c526:e507:e4c4:579b) Quit ()
[8:34] * JCL (~JCL@2601:9:5980:39b:c526:e507:e4c4:579b) has joined #ceph
[8:34] * JCL (~JCL@2601:9:5980:39b:c526:e507:e4c4:579b) Quit ()
[8:36] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has joined #ceph
[8:37] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[8:37] * zidarsk8 (~zidar@tm.78.153.58.217.dc.cable.static.telemach.net) has left #ceph
[8:42] * mcms (~mcms@46.224.152.64) has joined #ceph
[8:45] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[8:52] * madkiss (~madkiss@089144210096.atnat0019.highway.a1.net) has joined #ceph
[8:52] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[8:54] * mcms (~mcms@46.224.152.64) Quit (Quit: Leaving)
[8:57] * glambert (~glambert@37.157.50.80) Quit (Ping timeout: 480 seconds)
[9:01] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[9:08] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[9:17] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[9:24] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[9:24] * madkiss (~madkiss@089144210096.atnat0019.highway.a1.net) Quit (Quit: Leaving.)
[9:32] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[9:40] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[9:48] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[9:48] * jks (~jks@3e6b5724.rev.stofanet.dk) Quit (Quit: jks)
[9:55] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[9:57] <aarontc> how is it that 'ceph -s' can have such absurdly wrong values?
[9:57] <aarontc> recovery io 94131 MB/s, 23724 objects/s
[9:57] <aarontc> client io 19954 MB/s rd, 27724 MB/s wr, 282 kop/s
[10:04] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[10:05] * houkouonchi-home (~linux@pool-108-38-63-234.lsanca.fios.verizon.net) Quit (Remote host closed the connection)
[10:06] * glambert (~glambert@37.157.50.80) has joined #ceph
[10:09] * Meths_ (~meths@2.25.214.104) has joined #ceph
[10:10] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[10:11] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[10:12] * zack_dolby (~textual@p852cae.tokynt01.ap.so-net.ne.jp) has joined #ceph
[10:14] * glambert (~glambert@37.157.50.80) Quit (Ping timeout: 480 seconds)
[10:16] * Meths (~meths@2.25.213.161) Quit (Ping timeout: 480 seconds)
[10:19] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[10:21] * Kioob (~kioob@2a01:e34:ec0a:c0f0:21e:8cff:fe07:45b6) Quit (Quit: Leaving.)
[10:27] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[10:30] * Cataglottism (~Cataglott@dsl-087-195-030-184.solcon.nl) has joined #ceph
[10:34] * dalgaaf (uid15138@id-15138.ealing.irccloud.com) has joined #ceph
[10:35] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[10:36] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[10:42] * Cataglottism (~Cataglott@dsl-087-195-030-184.solcon.nl) Quit (Quit: My Mac Pro has gone to sleep. ZZZzzz???)
[10:43] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[10:51] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[10:56] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) has joined #ceph
[10:58] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[10:59] * jks (~jks@3e6b5724.rev.stofanet.dk) has joined #ceph
[11:06] * lofejndif (~lsqavnbok@tor-exit0-readme.ign.as) has joined #ceph
[11:06] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[11:07] * LeaChim (~LeaChim@host86-162-1-71.range86-162.btcentralplus.com) has joined #ceph
[11:14] * topro (~prousa@host-62-245-142-50.customer.m-online.net) has joined #ceph
[11:14] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[11:21] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[11:22] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[11:30] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[11:35] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[11:38] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[11:45] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[11:53] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[12:01] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[12:02] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[12:05] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit ()
[12:09] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[12:09] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[12:13] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit ()
[12:17] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[12:18] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[12:19] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[12:25] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[12:32] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[12:36] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) Quit (Remote host closed the connection)
[12:40] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[12:46] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[12:48] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[12:56] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[13:04] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[13:12] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[13:15] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) has joined #ceph
[13:19] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[13:28] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[13:33] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[13:35] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[13:35] * leseb (~leseb@185.21.174.206) Quit (Killed (NickServ (Too many failed password attempts.)))
[13:38] * leseb (~leseb@185.21.174.206) has joined #ceph
[13:43] * carif (~mcarifio@host200-164-static.240-95-b.business.telecomitalia.it) has joined #ceph
[13:43] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[13:47] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[13:50] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[13:51] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[13:59] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[14:03] * Sysadmin88 (~IceChat77@176.254.32.31) Quit (Quit: Always try to be modest, and be proud about it!)
[14:04] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) has joined #ceph
[14:07] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[14:16] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[14:23] * carif (~mcarifio@host200-164-static.240-95-b.business.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[14:23] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[14:28] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[14:30] <bjornar> What kind of iops can I expect from maildir load running on rbd?
[14:31] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[14:33] <bjornar> thinking of a 5x20osd dual 10gbe (5:1)
[14:39] * BillK (~BillK-OFT@106-69-172-170.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[14:39] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[14:40] * BillK (~BillK-OFT@106-69-213-97.dyn.iinet.net.au) has joined #ceph
[14:48] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[14:55] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[15:03] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[15:09] * Meths_ is now known as Meths
[15:11] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[15:14] * lxo (~aoliva@lxo.user.oftc.net) Quit (Quit: later)
[15:19] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[15:27] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[15:35] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[15:37] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) has joined #ceph
[15:37] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[15:38] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Remote host closed the connection)
[15:42] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[15:42] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[15:50] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[15:53] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) Quit (Ping timeout: 480 seconds)
[15:57] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[15:57] * analbeard (~shw@host31-53-108-38.range31-53.btcentralplus.com) has joined #ceph
[15:58] * analbeard (~shw@host31-53-108-38.range31-53.btcentralplus.com) Quit ()
[15:58] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[15:59] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[16:01] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Read error: Connection reset by peer)
[16:04] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[16:06] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[16:07] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[16:14] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[16:23] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[16:29] * carif (~mcarifio@host200-164-static.240-95-b.business.telecomitalia.it) has joined #ceph
[16:30] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[16:30] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[16:31] * andrein_ (~andrein@84.247.84.22) Quit (Read error: Connection reset by peer)
[16:32] * andrein_ (~andrein@84.247.84.22) has joined #ceph
[16:36] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[16:37] * kiwigeraint (~kiwigerai@208.72.139.54) has joined #ceph
[16:39] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[16:40] * kiwigeraint (~kiwigerai@208.72.139.54) Quit (Read error: Connection reset by peer)
[16:40] * kiwigera_ (~kiwigerai@208.72.139.54) has joined #ceph
[16:41] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[16:43] * kiwigera_ (~kiwigerai@208.72.139.54) Quit (Remote host closed the connection)
[16:47] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[16:47] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[16:47] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[16:53] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[16:55] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[16:56] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[17:02] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[17:03] * BillK (~BillK-OFT@106-69-213-97.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[17:09] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[17:09] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[17:10] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[17:17] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[17:18] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[17:25] * carif (~mcarifio@host200-164-static.240-95-b.business.telecomitalia.it) Quit (Quit: Ex-Chat)
[17:26] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[17:30] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[17:34] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[17:36] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[17:42] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[17:43] * lofejndif (~lsqavnbok@82JAADMV8.tor-irc.dnsbl.oftc.net) Quit (Remote host closed the connection)
[17:49] * andrein_ (~andrein@84.247.84.22) Quit (Ping timeout: 480 seconds)
[17:49] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[17:49] * stiopa (~stiopa@cpc11-dals19-2-0-cust132.20-2.cable.virginm.net) has joined #ceph
[17:50] * keoki_niu (~bufomarin@74-61-5-143.war.clearwire-wmx.net) has joined #ceph
[17:50] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[17:53] * keoki_niu (~bufomarin@74-61-5-143.war.clearwire-wmx.net) has left #ceph
[17:56] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[17:56] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[17:56] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Read error: Connection reset by peer)
[17:58] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[18:00] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[18:02] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[18:04] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[18:06] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[18:12] * fdmanana (~fdmanana@bl13-138-188.dsl.telepac.pt) has joined #ceph
[18:14] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[18:21] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[18:28] * Sysadmin88 (~IceChat77@176.254.32.31) has joined #ceph
[18:30] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[18:30] * andrein_ (~andrein@46.108.33.138) has joined #ceph
[18:37] * stiopa (~stiopa@cpc11-dals19-2-0-cust132.20-2.cable.virginm.net) Quit (Remote host closed the connection)
[18:37] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[18:37] * stiopa (stiopa@cpc11-dals19-2-0-cust132.20-2.cable.virginm.net) has joined #ceph
[18:37] * stiopa (stiopa@cpc11-dals19-2-0-cust132.20-2.cable.virginm.net) Quit ()
[18:45] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[18:47] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[18:53] * mkutsevol (~max@169-135-146-194.user.iteam.ua) has joined #ceph
[18:53] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[18:59] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) has joined #ceph
[19:01] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[19:02] * kiwigeraint (~kiwigerai@u74-116-251-050.sjc01.appliedops.net) has joined #ceph
[19:03] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[19:04] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[19:07] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) has joined #ceph
[19:08] * kiwigera_ (~kiwigerai@162.249.184.66) has joined #ceph
[19:09] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[19:14] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[19:14] * kiwigeraint (~kiwigerai@u74-116-251-050.sjc01.appliedops.net) Quit (Ping timeout: 480 seconds)
[19:15] * thuc (~thuc@c-71-198-202-49.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[19:18] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[19:18] * goerk (~goerk@ip-109-90-149-123.unitymediagroup.de) Quit (Read error: Connection reset by peer)
[19:22] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[19:23] * nwat (~textual@eduroam-237-164.ucsc.edu) has joined #ceph
[19:24] * madkiss (~madkiss@chello080108036100.31.11.vie.surfer.at) has joined #ceph
[19:25] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[19:27] * kiwigera_ (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[19:28] * Hasann (~horasan@88.234.38.234) has joined #ceph
[19:28] * Hasann (~horasan@88.234.38.234) Quit (autokilled: Do not spam other people. Mail support@oftc.net if you feel this is in error. (2014-04-05 17:28:06))
[19:28] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[19:30] * mozg (~andrei@46.229.149.194) has joined #ceph
[19:33] * chrisk (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[19:38] * madkiss (~madkiss@chello080108036100.31.11.vie.surfer.at) Quit (Quit: Leaving.)
[19:38] * Sysadmin88 (~IceChat77@176.254.32.31) Quit (Quit: Pull the pin and count to what?)
[19:40] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[19:41] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[19:41] * leseb (~leseb@185.21.174.206) Quit (Killed (NickServ (Too many failed password attempts.)))
[19:41] * leseb (~leseb@185.21.174.206) has joined #ceph
[19:41] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has joined #ceph
[19:41] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[19:41] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has left #ceph
[19:42] * fdmanana (~fdmanana@bl13-138-188.dsl.telepac.pt) Quit (Quit: Leaving)
[19:43] * Sysadmin88 (~IceChat77@176.254.32.31) has joined #ceph
[19:45] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[19:49] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[19:57] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[20:05] * bjornar (~bjornar@ti0099a430-0023.bb.online.no) Quit (Ping timeout: 480 seconds)
[20:05] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[20:05] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[20:12] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[20:15] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[20:19] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit ()
[20:19] * Boltsky (~textual@cpe-198-72-138-106.socal.res.rr.com) Quit (Quit: Textual IRC Client: www.textualapp.com)
[20:21] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[20:28] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[20:36] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[20:44] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[20:45] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[20:45] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[20:46] * scuttlemonkey (~scuttlemo@c-107-5-193-244.hsd1.mi.comcast.net) Quit (Quit: my troubles seem so far away, now yours are too...)
[20:46] * Daviey (~DavieyOFT@bootie.daviey.com) Quit (Ping timeout: 480 seconds)
[20:49] * scuttlemonkey (~scuttlemo@c-107-5-193-244.hsd1.mi.comcast.net) has joined #ceph
[20:49] * ChanServ sets mode +o scuttlemonkey
[20:52] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[20:56] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[20:59] * Daviey (~DavieyOFT@bootie.daviey.com) has joined #ceph
[21:00] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[21:08] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[21:10] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[21:15] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[21:15] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[21:18] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[21:19] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[21:24] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[21:28] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[21:29] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) has joined #ceph
[21:31] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[21:38] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[21:39] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[21:43] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) Quit (Remote host closed the connection)
[21:44] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) has joined #ceph
[21:47] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[21:50] * fdmanana (~fdmanana@bl13-138-188.dsl.telepac.pt) has joined #ceph
[21:51] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[21:54] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) Quit (Remote host closed the connection)
[21:55] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[22:01] * wonko_be (bernard@november.openminds.be) Quit (Read error: Operation timed out)
[22:02] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[22:07] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[22:08] * wonko_be (bernard@november.openminds.be) has joined #ceph
[22:08] * cbob (~cbob@host-63-232-9-69.midco.net) Quit (Ping timeout: 480 seconds)
[22:11] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[22:13] * mkutsevol_ (~max@188.190.207.111) has joined #ceph
[22:16] <mkutsevol_> Need help. I'm new to ceph. I configured a cluster with 3mons+2ods. With default pools (rbd, data, metadata). Then I recreated ODSes. loosing all data - I took both ODSes down and reformatted their data partitions and then deleted then from configuration and readded with new keyrings.
[22:17] <mkutsevol_> Now I have cluster in state: health HEALTH_WARN 128 pgs degraded; 128 pgs stale; 128 pgs stuck stale; 128 pgs stuck unclean. (I deleted pool rbd, there was 192 stuck pgs in the beginning)
[22:17] <mkutsevol_> How can I tell ceph that these pgs will never come back?
[22:18] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[22:18] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[22:19] <mikedawson> mkutsevol_: can you paste the output of 'ceph osd tree' and 'ceph osd dump | grep ^pool'?
[22:19] <mkutsevol_> yes,
[22:19] <mkutsevol_> ceph> osd tree
[22:19] <mkutsevol_> # id weight type name up/down reweight
[22:19] <mkutsevol_> -1 40 root default
[22:19] <mkutsevol_> -2 20 host osd0
[22:19] <mkutsevol_> 0 20 osd.0 up 1
[22:19] <mkutsevol_> -3 20 host osd1
[22:19] <mkutsevol_> 1 20 osd.1 up 1
[22:20] <mkutsevol_> pool 0 'data' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 flags hashpspool crash_replay_interval 45 stripe_width 0
[22:20] <mkutsevol_> pool 1 'metadata' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 flags hashpspool stripe_width 0
[22:22] * Karcaw_ (~evan@96-41-200-66.dhcp.elbg.wa.charter.com) Quit (Ping timeout: 480 seconds)
[22:22] <mikedawson> you have two pools with 64 PGs each, so reporting 128 PGs is right. Also, both pools have a size of 2, so your two hosts should each get a copy with default crush rules. Unfortunately, none of these PGs seems to be replicating correctly.
[22:23] <mikedawson> Can you paste the output of 'ceph health detail' somewhere, like fpaste or pastebin?
[22:23] <mkutsevol_> yes
[22:25] <mkutsevol_> http://paste.ubuntu.com/7209445/
[22:26] <mkutsevol_> I was severely incorrect with ceph - I first killed all OSDs, then reformatted their data partitions and added theb back with the same ids. Does it matter?
[22:26] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[22:27] <mikedawson> mkutsevol_: look at the 'last acting [....]' part on each line. Because you have 2x replication on your pools, each PG should get placed on [0,1] or [1,0] where the first osd listed is the primary.
[22:29] <mikedawson> mkutsevol_: I think you are best to start over from scratch. Of if you'd like to learn more and attempt to recover from this state, remove both pools, then remove osd.0 and osd.1. After that, you could re-add the two osds. They'll get named osd.0 and osd.1.
[22:29] <mkutsevol_> last acting [3] - it was osd.3 ?
[22:29] <mikedawson> Then re-add the pools. Not sure if that will work, but you may learn something along the way. Like I said, you should probably start from scratch at this point.
[22:30] <mikedawson> Yes, last acting [3] implies there was at one point an osd.3
[22:30] <mkutsevol_> yes, I had osd.3
[22:31] <mkutsevol_> these are default pools - data and metadata when I try to remove them I get such message:
[22:32] <mkutsevol_> ceph> osd pool delete data data --yes-i-really-really-mean-it
[22:32] <mkutsevol_> Error: 16 EBUSY
[22:32] <mkutsevol_> Status:
[22:32] <mkutsevol_> pool 'data' is in use by CephFS
[22:32] <mkutsevol_> ceph>
[22:32] <mkutsevol_> but I cant find where I have CephFS
[22:34] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[22:42] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[22:43] <mkutsevol_> Question about journal: i manually create symlink to cache partition from data directory after executing: --mkfs --mkjournal --osd-journal=/dev/vdc1
[22:43] <mkutsevol_> is it the right way?
[22:47] * lofejndif (~lsqavnbok@spftor5e1.privacyfoundation.ch) has joined #ceph
[22:50] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[22:57] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) has joined #ceph
[22:58] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[23:05] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) has joined #ceph
[23:07] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[23:07] * kiwigeraint (~kiwigerai@162.249.184.66) has joined #ceph
[23:08] <mkutsevol_> pg 1.2c is stuck unclean for 24171.718963, current state stale+active+degraded, last acting [3]
[23:08] <mkutsevol_> but
[23:08] <mkutsevol_> ceph> pg scrub 1.2c
[23:08] <mkutsevol_> Error: 11 EAGAIN
[23:08] <mkutsevol_> Status:
[23:08] <mkutsevol_> pg 1.2c primary osd.3 not up
[23:09] <mkutsevol_> if I do not want to bring osd.3 back - is there a way to do something with pg?
[23:13] * ckranz (~Chris@cpc26-salf5-2-0-cust77.10-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[23:14] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[23:15] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) Quit (Read error: Connection reset by peer)
[23:16] * BManojlovic (~steki@cable-94-189-160-252.dynamic.sbb.rs) has joined #ceph
[23:17] * nwat (~textual@eduroam-237-164.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[23:20] * kiwigeraint (~kiwigerai@162.249.184.66) Quit (Remote host closed the connection)
[23:29] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[23:31] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit ()
[23:33] * Sysadmin88 (~IceChat77@176.254.32.31) Quit (Quit: ASCII a stupid question, get a stupid ANSI!)
[23:45] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[23:48] * Karcaw (~evan@96-41-200-66.dhcp.elbg.wa.charter.com) has joined #ceph
[23:58] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.