#ceph IRC Log

Index

IRC Log for 2014-06-08

Timestamps are in GMT/BST.

[0:01] * nwat (~textual@eduroam-249-245.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[0:14] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[0:15] * sarob (~sarob@2601:9:1d00:c7f:69ae:4841:67cf:bcb4) has joined #ceph
[0:17] * Sysadmin88 (~IceChat77@94.4.22.173) Quit (Quit: I used to think I was indecisive, but now I'm not too sure.)
[0:17] * Sysadmin88 (~IceChat77@94.4.22.173) has joined #ceph
[0:23] * sarob (~sarob@2601:9:1d00:c7f:69ae:4841:67cf:bcb4) Quit (Ping timeout: 480 seconds)
[0:41] * diegows (~diegows@190.190.5.238) has joined #ceph
[0:57] * gillesMo (~gillesMo@00012912.user.oftc.net) has joined #ceph
[1:03] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[1:05] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[1:14] * joerocklin (~joe@cpe-75-186-9-154.cinci.res.rr.com) Quit (Remote host closed the connection)
[1:15] * sarob (~sarob@2601:9:1d00:c7f:fc80:723a:53a2:7818) has joined #ceph
[1:19] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[1:20] * joerocklin (~joe@cpe-75-186-9-154.cinci.res.rr.com) has joined #ceph
[1:21] * oms101 (~oms101@p20030057EA29CD00EEF4BBFFFE0F7062.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[1:23] * sarob (~sarob@2601:9:1d00:c7f:fc80:723a:53a2:7818) Quit (Ping timeout: 480 seconds)
[1:29] * oms101 (~oms101@p20030057EA474B00EEF4BBFFFE0F7062.dip0.t-ipconnect.de) has joined #ceph
[1:32] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) Quit (Read error: Operation timed out)
[1:35] * zidarsk81 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[1:35] * zidarsk81 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[1:38] * jsfrerot (~jsfrerot@192.222.132.57) Quit (Quit: leaving)
[1:45] * rturk|afk is now known as rturk
[1:53] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[1:53] * gillesMo (~gillesMo@00012912.user.oftc.net) Quit (Quit: Konversation terminated!)
[1:55] * sarob_ (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[1:57] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) has joined #ceph
[2:00] * rturk is now known as rturk|afk
[2:01] * aldavud (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[2:01] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[2:03] * sarob_ (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[2:04] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Remote host closed the connection)
[2:04] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[2:12] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[2:14] * sherry (~sherry@mike-alien.esc.auckland.ac.nz) Quit (Remote host closed the connection)
[2:15] * sarob (~sarob@2601:9:1d00:c7f:84fa:11bf:f291:3104) has joined #ceph
[2:15] * rturk|afk is now known as rturk
[2:21] * sherry (~sherry@mike-alien.esc.auckland.ac.nz) has joined #ceph
[2:23] * sarob (~sarob@2601:9:1d00:c7f:84fa:11bf:f291:3104) Quit (Ping timeout: 480 seconds)
[2:26] <sherry> Hi, anyone here?
[2:26] * rturk is now known as rturk|afk
[2:38] * danieagle (~Daniel@179.186.127.228.dynamic.adsl.gvt.net.br) Quit (Quit: Obrigado por Tudo! :-) inte+ :-))
[2:42] * Cube (~Cube@66-87-130-64.pools.spcsdns.net) has joined #ceph
[2:52] * JayJ (~jayj@pool-96-233-113-153.bstnma.fios.verizon.net) has joined #ceph
[3:07] * LeaChim (~LeaChim@host86-174-77-240.range86-174.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[3:12] * JayJ (~jayj@pool-96-233-113-153.bstnma.fios.verizon.net) Quit (Quit: Computer has gone to sleep.)
[3:12] * jsfrerot (~jsfrerot@192.222.132.57) has joined #ceph
[3:13] <jsfrerot> hi, got a few questions tonight if someone is available to answer.
[3:14] <jsfrerot> I'm trying to find a way to back my ceph cluster. What is the way to go. I've seen some thread about that, but this seemed to be kind of experimental. Is there any "official" way to do this
[3:14] <jsfrerot> What I mean to do is to restore data that have been deleted from the cluster at some point.
[3:15] * JayJ (~jayj@pool-96-233-113-153.bstnma.fios.verizon.net) has joined #ceph
[3:15] * sarob (~sarob@2601:9:1d00:c7f:e976:d003:4302:462c) has joined #ceph
[3:16] <jsfrerot> another question I have is, how can I know which version of the mon,osd and mds that is currently running on the different nodes of my cluster?
[3:16] <jsfrerot> Let say i'm doing a rolling upgrade, I'd like to know which one have been updated, and which one are left to do.
[3:17] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[3:19] * JayJ (~jayj@pool-96-233-113-153.bstnma.fios.verizon.net) Quit ()
[3:23] * sarob (~sarob@2601:9:1d00:c7f:e976:d003:4302:462c) Quit (Ping timeout: 480 seconds)
[3:25] <jsfrerot> exit
[3:25] * jsfrerot (~jsfrerot@192.222.132.57) Quit (Quit: leaving)
[3:26] * jsfrerot (~jsfrerot@192.222.132.57) has joined #ceph
[3:38] * haomaiwang (~haomaiwan@124.161.77.90) Quit (Ping timeout: 480 seconds)
[3:46] * diegows (~diegows@190.190.5.238) Quit (Ping timeout: 480 seconds)
[4:00] * rturk|afk is now known as rturk
[4:02] * rturk is now known as rturk|afk
[4:04] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[4:10] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[4:13] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[4:15] * sarob (~sarob@2601:9:1d00:c7f:c8d9:ee47:6e0c:fda9) has joined #ceph
[4:22] * haomaiwang (~haomaiwan@119.6.75.101) has joined #ceph
[4:23] * sarob (~sarob@2601:9:1d00:c7f:c8d9:ee47:6e0c:fda9) Quit (Ping timeout: 480 seconds)
[4:27] * haomaiwa_ (~haomaiwan@119.6.75.101) has joined #ceph
[4:30] * haomaiwang (~haomaiwan@119.6.75.101) Quit (Ping timeout: 480 seconds)
[4:32] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) has joined #ceph
[4:38] * haomaiwa_ (~haomaiwan@119.6.75.101) Quit (Read error: Connection reset by peer)
[4:39] * haomaiwang (~haomaiwan@112.193.129.69) has joined #ceph
[4:43] * haomaiwang (~haomaiwan@112.193.129.69) Quit (Read error: Connection reset by peer)
[4:44] * haomaiwang (~haomaiwan@112.193.129.254) has joined #ceph
[5:02] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[5:03] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[5:08] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[5:12] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[5:13] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) Quit (Quit: Leaving.)
[5:15] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[5:15] * sarob (~sarob@2601:9:1d00:c7f:31a2:7965:4921:b122) has joined #ceph
[5:18] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[5:23] * sarob (~sarob@2601:9:1d00:c7f:31a2:7965:4921:b122) Quit (Ping timeout: 480 seconds)
[5:26] * aldavud (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[5:33] * Vacum (~vovo@88.130.221.184) has joined #ceph
[5:40] * Vacum_ (~vovo@i59F7A54D.versanet.de) Quit (Ping timeout: 480 seconds)
[5:55] * The_Bishop (~bishop@2001:470:50b6:0:3864:48e6:fc79:b7fb) has joined #ceph
[6:02] * erice_ (~erice@c-98-245-48-79.hsd1.co.comcast.net) Quit (Remote host closed the connection)
[6:02] * haomaiwang (~haomaiwan@112.193.129.254) Quit (Read error: Connection reset by peer)
[6:05] * oblu (~o@62.109.134.112) Quit (Quit: ~)
[6:12] * oblu (~o@62.109.134.112) has joined #ceph
[6:15] * sarob (~sarob@2601:9:1d00:c7f:618c:bb60:1379:c6ae) has joined #ceph
[6:15] * haomaiwang (~haomaiwan@124.161.76.148) has joined #ceph
[6:16] * haomaiwang (~haomaiwan@124.161.76.148) Quit (Remote host closed the connection)
[6:17] * haomaiwang (~haomaiwan@124.248.205.17) has joined #ceph
[6:23] * sarob (~sarob@2601:9:1d00:c7f:618c:bb60:1379:c6ae) Quit (Ping timeout: 480 seconds)
[6:32] * haomaiwa_ (~haomaiwan@124.161.76.148) has joined #ceph
[6:36] * haomaiwang (~haomaiwan@124.248.205.17) Quit (Ping timeout: 480 seconds)
[6:44] * nwat (~textual@50.141.87.6) has joined #ceph
[6:47] * nwat_ (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[6:47] * The_Bishop (~bishop@2001:470:50b6:0:3864:48e6:fc79:b7fb) Quit (Ping timeout: 480 seconds)
[6:50] * yuriw1 (~Adium@AMontpellier-653-1-321-172.w86-202.abo.wanadoo.fr) has joined #ceph
[6:52] * nwat (~textual@50.141.87.6) Quit (Ping timeout: 480 seconds)
[6:54] * nwat_ (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[6:55] * yuriw (~Adium@AMontpellier-653-1-330-229.w86-202.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[6:57] * wschulze (~wschulze@cpe-69-206-251-158.nyc.res.rr.com) Quit (Quit: Leaving.)
[6:58] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[7:07] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) Quit (Quit: Leaving.)
[7:11] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[7:14] <sherry> anyone here can tell me why removing files in CephFS, does not remove files objects from osds!
[7:15] * sarob (~sarob@2601:9:1d00:c7f:d48d:97d0:727d:8d45) has joined #ceph
[7:23] * sarob (~sarob@2601:9:1d00:c7f:d48d:97d0:727d:8d45) Quit (Ping timeout: 480 seconds)
[7:25] * Cube (~Cube@66-87-130-64.pools.spcsdns.net) Quit (Quit: Leaving.)
[7:33] * dcurtiss_ (~dcurtiss@130.164.62.72) has joined #ceph
[7:33] * dcurtiss (~dcurtiss@130.164.62.72) Quit (Read error: Connection reset by peer)
[7:57] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) has joined #ceph
[8:04] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[8:06] * The_Bishop (~bishop@e178114196.adsl.alicedsl.de) has joined #ceph
[8:15] * sarob (~sarob@2601:9:1d00:c7f:60b5:647b:17b0:d63e) has joined #ceph
[8:17] * haomaiwa_ (~haomaiwan@124.161.76.148) Quit (Remote host closed the connection)
[8:17] * haomaiwang (~haomaiwan@124.248.205.17) has joined #ceph
[8:23] * sarob (~sarob@2601:9:1d00:c7f:60b5:647b:17b0:d63e) Quit (Ping timeout: 480 seconds)
[8:23] * haomaiwa_ (~haomaiwan@124.161.76.148) has joined #ceph
[8:26] * haomaiwang (~haomaiwan@124.248.205.17) Quit (Ping timeout: 480 seconds)
[8:33] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[8:33] * yuriw1 (~Adium@AMontpellier-653-1-321-172.w86-202.abo.wanadoo.fr) has left #ceph
[8:33] * simulx (~simulx@vpn.expressionanalysis.com) Quit (Ping timeout: 480 seconds)
[8:34] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[8:36] * rendar (~I@87.19.182.19) has joined #ceph
[9:10] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[9:15] * sarob_ (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[9:18] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[9:23] * sarob_ (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[9:25] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[9:30] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[9:31] * The_Bishop (~bishop@e178114196.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[9:32] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[9:40] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) Quit (Remote host closed the connection)
[9:43] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Remote host closed the connection)
[9:43] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[9:46] * zligor (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) has joined #ceph
[9:47] * zligor (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) Quit (Read error: Connection reset by peer)
[9:48] * Nacer_ (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[9:48] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Read error: Connection reset by peer)
[9:51] * imriz (~imriz@82.81.163.130) has joined #ceph
[9:52] <imriz> Hi, I want to upgrade from 0.67.7 to .9. My cluster is doing recovery right now - Is that a big no-no to do the upgrade while the cluster is recovering?
[9:56] * cookednoodles (~eoin@eoin.clanslots.com) has joined #ceph
[9:57] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[10:01] * zligor (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) has joined #ceph
[10:02] * zligor (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) Quit (Read error: Connection reset by peer)
[10:04] * ikrstic (~ikrstic@77-46-245-216.dynamic.isp.telekom.rs) Quit (Quit: Konversation terminated!)
[10:04] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) has joined #ceph
[10:05] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) Quit (Read error: Connection reset by peer)
[10:07] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[10:14] * fdmanana (~fdmanana@bl10-253-137.dsl.telepac.pt) has joined #ceph
[10:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[10:21] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[10:23] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[10:25] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[10:26] * Cube (~Cube@66-87-130-64.pools.spcsdns.net) has joined #ceph
[10:31] * Cube (~Cube@66-87-130-64.pools.spcsdns.net) Quit ()
[10:33] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[10:33] * Cube (~Cube@66.87.130.64) has joined #ceph
[10:34] * Cube (~Cube@66.87.130.64) Quit ()
[10:47] * haomaiwang (~haomaiwan@124.248.205.17) has joined #ceph
[10:47] * haomaiwa_ (~haomaiwan@124.161.76.148) Quit (Read error: Connection reset by peer)
[10:56] * haomaiwa_ (~haomaiwan@124.161.76.148) has joined #ceph
[11:03] * haomaiwang (~haomaiwan@124.248.205.17) Quit (Ping timeout: 480 seconds)
[11:07] * haomaiwang (~haomaiwan@li671-185.members.linode.com) has joined #ceph
[11:08] * m0e (~Moe@41.45.48.221) has joined #ceph
[11:08] * fred` is now known as fred
[11:09] * fred is now known as fred`
[11:10] * haomaiwa_ (~haomaiwan@124.161.76.148) Quit (Read error: Operation timed out)
[11:12] * fred` is now known as [fred]
[11:12] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) has joined #ceph
[11:13] * toabctl (~toabctl@static.163.127.9.5.clients.your-server.de) Quit (Quit: Adios)
[11:13] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) Quit (Read error: Connection reset by peer)
[11:14] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) has joined #ceph
[11:14] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) Quit (Read error: Connection reset by peer)
[11:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[11:16] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[11:16] * toabctl (~toabctl@toabctl.de) has joined #ceph
[11:23] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[11:23] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) has joined #ceph
[11:24] * ikrstic (~ikrstic@net253-0-245-109.mbb.telenor.rs) has left #ceph
[11:26] * ultimape (~Ultimape@c-174-62-192-41.hsd1.vt.comcast.net) has joined #ceph
[11:27] <ultimape> Hello! Is anyone awake and willing to help a newbie?
[11:31] <ultimape> I have ubuntu server installed on a thumbdrive, and I'd like to hook up both internal and external drives to it and have ceph share them as block devices or smb shares. I'm getting a little lost on the terminology. The documentation refers to nodes as indidivual machines - is it possible to get a one machine system up and running?
[11:33] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[11:43] <cookednoodles> yes, but it defeats the point
[11:43] <ultimape> Iv'e got a bunch of pcs I'm hoping to add the the cluster, but I don't want to bite off more than I can chew.
[11:44] <cookednoodles> the thing is, its a lot more complicated to make a 1 node cluster than a 3 node one
[11:46] <ultimape> Right now I've got some stuff running on the metal. I need to bootstrap something together before I can turn them into VMs and free up the hardware.
[11:46] <cookednoodles> how many drives are you going to use for ceph on your 1 machine setup ?
[11:48] <ultimape> Well, I can reliably get 8 hooked up to the machine and visible from ubuntu. 5 of them are externals that I would like to move to the other machines once I can free them up.
[11:48] <ultimape> I think minimum I'd need 3 'osd's for the 3 internal drives.
[11:49] <cookednoodles> +journals if you want performance
[11:49] <cookednoodles> if you put the journal and storage on the same drive, you cut your performance in hald
[11:49] <cookednoodles> *half
[11:50] <ultimape> is there documentation on journals?
[11:52] * nljmo (~nljmo@5ED6C263.cm-7-7d.dynamic.ziggo.nl) has joined #ceph
[11:54] <cookednoodles> probably somewhere :P
[11:55] <ultimape> it says it needs a journal, but not much else. does that mean 2 drives per OSD daemon, or can they share a journal?
[11:55] * hitsumabushi (~hitsumabu@nttkyo215214.tkyo.nt.ngn2.ppp.infoweb.ne.jp) has joined #ceph
[11:57] * steveeJ (~junky@HSI-KBW-46-223-54-149.hsi.kabel-badenwuerttemberg.de) has joined #ceph
[12:01] * aldavud (~aldavud@212.243.10.250) Quit (Quit: leaving)
[12:06] * Nacer_ (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Remote host closed the connection)
[12:06] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[12:07] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[12:14] <ultimape> I think I can deal with the double write hit for now, I'm not toing to be write heavy. If it becomes an issue it looks like I should be able to redirect the journal at a later time.
[12:14] <ultimape> ( http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-June/002383.html )
[12:15] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[12:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[12:15] <sherry> ultimape: as far I as I know the purpose of journal is improving the performance of I/O, by sharing the journal for more than one OSD u get no point.
[12:18] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) has joined #ceph
[12:18] <ultimape> Based on this: http://www.sebastien-han.fr/blog/2014/02/17/ceph-io-patterns-the-bad/ I think I should be fine if I get a usb 3 card and stick in some fast thumbdrives that work well with window's ReadyBoost. I can't imagine the write rate being any higher than the NIC card can support anyway.
[12:20] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[12:20] <cookednoodles> fast thumbdrive ? thats an oxymoron :P
[12:20] <cookednoodles> you want iops, thats not something they are good at
[12:20] <cookednoodles> but it all depends on what you are after
[12:31] <ultimape> So what exactly would I need if I have 2x1tb hdds and OS on a thumbdrive. 1 mon, and 2 OSDs?
[12:33] <ultimape> what exactly is an 'admin' node? Is that just a box that can ssh to the actual cluster?
[12:35] * ghartz_ (~ghartz@ip-68.net-80-236-84.joinville.rev.numericable.fr) has joined #ceph
[12:40] * finster (~finster@cmdline.guru) Quit (Quit: leaving)
[12:41] <cookednoodles> I'd put the journal on the hdd, you'll lose performance, but you don't want your os and journals on a usb key
[12:41] <cookednoodles> and yes to the admin node
[12:43] <ultimape> I'm planning on treating the usb key like I did with my old EEEpc - redirect anything that is write heavy to a real drive (/tmp etc). For right now it will do in the short term.
[12:44] <ultimape> my understanding is that I can pop out a drive (after turning off it's OSD) and have it hooked up to another machine/OSD and it just figures itself out?
[12:46] <Vacum> ultimape: yes, but depending on the crushmap and crushrule of your pools this will very likely result in placement groups being shuffled around between the OSDs
[12:47] * m0e (~Moe@41.45.48.221) Quit (Quit: This computer has gone to sleep)
[12:47] <Vacum> ultimape: also your idea of starting with a single machine and 2 OSDs isn't "future compatible" with later adding more machines. to make your setup work you will need to modify firefly's default crushrule
[12:49] <Vacum> with 2 osds in the same machine your crushrule will need to pick its leafs from on the OSD level (default is on host level). if you later add in more hosts and you keep that crushrule, you will end up with replicas of the same pgs on osds of the same host. which defeats the purpose
[12:49] <Vacum> but if you later change the crushrule, a lot of pgs will be moved around in your cluster
[12:51] <Vacum> ultimape: for a short, small "lets see if ceph works at all for me" test your setup is ok. but for anything else, ie failure tests, performance its not a good idea
[12:51] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[12:52] <Vacum> with your setup you can ie test if the ceph "clients" (rgw, cephfs, rbd) are working OK for you - but its not suited to evaluate the "cluster side" of ceph: rados itself and crush
[12:52] <ultimape> will all that moving around sort it self out after a time, or will changing the crush rule cause permanent damage?
[12:53] <ultimape> or are you saying going from 1 machine -> 3 machines will be impossible?
[12:54] <Vacum> ultimape: no, its not impossible. and losing data is very unlikely by this. but depending on the amount of data already stored and your hardware (network, iops, ram) it can take a long time
[12:55] <Vacum> ultimape: adding more machines is a normal process for ceph. but you are creating an edge case here with 1 machine
[12:55] <Vacum> ultimape: if you would start with 3 machines, 1 osd each, that would already fit the default values of firefly and "just work". if you then later added more machines or more osds to the existing machines, no changes to the crushrules are needed
[12:56] <Vacum> ultimape: yet, each time you add or remove an osd, of course data has to move
[12:57] <Vacum> ultimape: but in your case you would need to change the crushrule of existing pools, which (to my understanding) will result to more data that has to be moved
[12:57] <ultimape> where would I find 'firefly'. I'd like to see it to get come context.
[12:58] <Vacum> ultimape: firefly is the release name of the current stable release. 0.80.1
[12:58] <ultimape> I guess it's a balancing act. The other alternative is to push all the data onto a temporary raid5 - but that is pretty slow in itself.
[12:58] <Vacum> ultimape: I don't yet understand what you are trying to achieve there? :)
[13:02] <ultimape> Right now I've got 3 machines and a bunch of external drives that have disparate bits of data at about 60% capacity - but not evenly distributed. I want to move the data to a ceph cluster so I can make it redundant and add to the storage as I go. The trick is that I've only been able to free up one machine for creating the cluster - the data was pushed to the other two machines/drives temporarily.
[13:02] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Read error: Operation timed out)
[13:03] <ultimape> I was hoping ot find a way to bootstrap my way there by starting with one machine and using it's now ceph-ified storage to free up the next machine etc.
[13:03] * The_Bishop (~bishop@e178114196.adsl.alicedsl.de) has joined #ceph
[13:03] <Vacum> ultimape: I strongly suggest you first try out ceph on test machines, not those productive machines. and get at least some understanding how it works. and if it actually matches your usecases
[13:05] <Vacum> ultimape: is that a home / leisure time project?
[13:05] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[13:05] * m0e (~Moe@41.45.48.221) has joined #ceph
[13:05] <ultimape> i guess you'd call it a home-lab. I'm looking to get a feel for it to help out a local hackerspace.
[13:07] <ultimape> Is it feasable to get up and running on 1x64bit machine and 2x32bit machines? I heard it was a limitation in a video, but can't find the docs to confirm it.
[13:07] <Vacum> ultimape: get 3 old machines somewhere (friends, ebay) and a bunch of old hdds to start it
[13:08] <Vacum> ultimape: mh, no idea about 32bit. even all my home machines are 64bit now :)
[13:09] <ultimape> I've got 16 shuttle pcs that are all 32 bit. I'd love to get them going in a cluster, but its silly if they won't work with the 64 bit machines that I've got.
[13:10] <Vacum> ultimape: mh, that would be a very good start :) sorry I have no insight about the 32bit question
[13:10] * Fetch (fetch@gimel.cepheid.org) Quit (Read error: Connection reset by peer)
[13:10] <mfa298> for testing things out you could even go for a bunch of vm's but that will obviously impact performance and limits any redundancy - but it's a decent way to learn how ceph works (it's how I'm currently experimenting with things)
[13:11] <ultimape> you've been helpful in anycase. It sounds like I'll have to save my 1 machine cluster expansion thing for when im' more experianced.
[13:12] * Fetch (fetch@gimel.cepheid.org) has joined #ceph
[13:12] <ultimape> I wish hexchat had spellcheck
[13:12] <ultimape> mfa298, what distro/vm are you using?
[13:13] <mfa298> Using vmware ESXi for the hypervisor then running vm's with ubuntu 14.04 for monitors and OSDs
[13:14] <Vacum> ultimape: i googled for 32bit and ceph a bit. it seems the major issue comes from using cephfs on 32bit machines. not sure how it works out if you use 32bit machines for the OSDs only
[13:14] <mfa298> I've just started trying to get the radosgw parts running and want to look at erasure pools and calamari etc. as well.
[13:15] * steveeJ (~junky@HSI-KBW-46-223-54-149.hsi.kabel-badenwuerttemberg.de) Quit (Remote host closed the connection)
[13:15] * sarob (~sarob@2601:9:1d00:c7f:a04a:1319:1ad4:69a3) has joined #ceph
[13:17] * dereky (~derek@209-6-36-147.c3-0.smr-ubr2.sbo-smr.ma.cable.rcn.com) has joined #ceph
[13:21] * The_Bishop_ (~bishop@f048189061.adsl.alicedsl.de) has joined #ceph
[13:22] * dereky (~derek@209-6-36-147.c3-0.smr-ubr2.sbo-smr.ma.cable.rcn.com) Quit (Quit: dereky)
[13:23] * sarob (~sarob@2601:9:1d00:c7f:a04a:1319:1ad4:69a3) Quit (Ping timeout: 480 seconds)
[13:25] * dereky (~derek@209-6-36-147.c3-0.smr-ubr2.sbo-smr.ma.cable.rcn.com) has joined #ceph
[13:27] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) Quit (Remote host closed the connection)
[13:28] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) has joined #ceph
[13:29] * The_Bishop (~bishop@e178114196.adsl.alicedsl.de) Quit (Ping timeout: 480 seconds)
[13:31] * simulx (~simulx@66-194-114-178.static.twtelecom.net) has joined #ceph
[13:35] * m0e (~Moe@41.45.48.221) Quit (Quit: This computer has gone to sleep)
[13:36] * dereky (~derek@209-6-36-147.c3-0.smr-ubr2.sbo-smr.ma.cable.rcn.com) Quit (Quit: dereky)
[13:41] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[13:53] * m0e (~Moe@41.45.48.221) has joined #ceph
[13:59] * LeaChim (~LeaChim@host86-174-77-240.range86-174.btcentralplus.com) has joined #ceph
[14:02] * m0e (~Moe@41.45.48.221) Quit (Quit: This computer has gone to sleep)
[14:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[14:18] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) has joined #ceph
[14:20] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) Quit ()
[14:23] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[14:25] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) has joined #ceph
[14:26] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) has joined #ceph
[14:28] * JayJ (~jayj@pool-96-233-113-153.bstnma.fios.verizon.net) has joined #ceph
[14:29] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) Quit ()
[14:30] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) has joined #ceph
[14:31] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) Quit ()
[14:31] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) has joined #ceph
[14:32] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) Quit ()
[14:32] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) has joined #ceph
[14:33] * tracphil_ (~tracphil@pool-71-163-168-251.washdc.fios.verizon.net) Quit ()
[14:39] * zz_hitsumabushi (~hitsumabu@175.184.30.148) has joined #ceph
[14:44] * zz_hitsumabushi (~hitsumabu@175.184.30.148) Quit (Quit: ZNC - http://znc.in)
[14:44] * zz_hitsumabushi (~hitsumabu@175.184.30.148) has joined #ceph
[14:47] * haomaiwang (~haomaiwan@li671-185.members.linode.com) Quit (Ping timeout: 480 seconds)
[14:50] * haomaiwang (~haomaiwan@119.4.215.167) has joined #ceph
[14:50] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[14:53] * haomaiwang (~haomaiwan@119.4.215.167) Quit (Remote host closed the connection)
[14:54] * haomaiwa_ (~haomaiwan@li671-185.members.linode.com) has joined #ceph
[14:58] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[15:10] * hitsumabushi (~hitsumabu@nttkyo215214.tkyo.nt.ngn2.ppp.infoweb.ne.jp) Quit (Quit: Good Bye :))
[15:11] * zz_hitsumabushi is now known as hitsumabushi
[15:13] * aldavud (~aldavud@212.243.10.250) Quit (Read error: Operation timed out)
[15:15] * sarob (~sarob@2601:9:1d00:c7f:50cd:4efb:e7de:3227) has joined #ceph
[15:22] * JayJ (~jayj@pool-96-233-113-153.bstnma.fios.verizon.net) Quit (Quit: Computer has gone to sleep.)
[15:23] * sarob (~sarob@2601:9:1d00:c7f:50cd:4efb:e7de:3227) Quit (Ping timeout: 480 seconds)
[15:24] * m0e (~Moe@41.45.48.221) has joined #ceph
[15:29] * tws_ (~traviss@rrcs-24-123-86-154.central.biz.rr.com) Quit (Ping timeout: 480 seconds)
[15:37] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[15:44] * nwat (~textual@50.141.87.2) has joined #ceph
[15:45] * nwat_ (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[15:45] * aldavud (~aldavud@212.243.10.250) Quit (Read error: Operation timed out)
[15:51] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[15:51] * haomaiwa_ (~haomaiwan@li671-185.members.linode.com) Quit (Remote host closed the connection)
[15:52] * haomaiwang (~haomaiwan@li671-185.members.linode.com) has joined #ceph
[15:52] * nwat (~textual@50.141.87.2) Quit (Ping timeout: 480 seconds)
[15:55] * oblu (~o@62.109.134.112) Quit (Quit: ~)
[15:59] * hitsumabushi is now known as zz_hitsumabushi
[16:01] * oblu (~o@62.109.134.112) has joined #ceph
[16:04] * zz_hitsumabushi (~hitsumabu@175.184.30.148) Quit (Quit: ZNC - http://znc.in)
[16:05] * zz_hitsumabushi (~hitsumabu@175.184.30.148) has joined #ceph
[16:06] * zz_hitsumabushi is now known as hitsumabushi
[16:09] <ultimape> in http://ceph.com/docs/dumpling/start/quick-ceph-deploy/ it refers to cuttlefish. Should I use firefly instead?
[16:13] <Vacum> ultimape: better check out the current version of that guide: http://ceph.com/docs/master/start/quick-ceph-deploy/
[16:13] <Vacum> ultimape: and imo you should use firefly from the start, yes
[16:14] <ultimape> dumpling lol
[16:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[16:15] <Vacum> ultimape: its http://www.thecephalopodpage.org/Etasmanica.php and not https://en.wikipedia.org/wiki/Dumpling :)
[16:16] <ultimape> I guess that explains part of my confusion - master vs dumpling etc
[16:16] <Vacum> ultimape: although you could possibly make the latter out of the former :)
[16:18] <ultimape> any of the releases named kraken yet?
[16:18] <Vacum> ultimape: no. they are sorted alphabetically. bobtail, cuttlefish, dumpling, emperor, firefly. next is giant
[16:20] <ultimape> hmm. well, TIL
[16:20] <ultimape> :)
[16:22] * aldavud_ (~aldavud@212.243.10.250) has joined #ceph
[16:22] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[16:50] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[16:54] * wschulze (~wschulze@cpe-69-206-251-158.nyc.res.rr.com) has joined #ceph
[16:58] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[16:58] * DV (~veillard@2001:41d0:1:d478::1) Quit (Remote host closed the connection)
[17:02] * DV (~veillard@libvirt.org) has joined #ceph
[17:07] * cookednoodles (~eoin@eoin.clanslots.com) Quit (Quit: Ex-Chat)
[17:07] * narb (~Jeff@38.99.52.10) has joined #ceph
[17:11] <ultimape> hmm, is it normal to receive a 'cluster may not be in healthy state' when doing "ceph-deploy mon create-initial" ? seems silly that it would even try
[17:12] <ultimape> re: http://ceph.com/docs/master/start/quick-ceph-deploy/ 'create cluster' step 5
[17:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[17:16] * Nacer (~Nacer@c2s31-2-83-152-89-219.fbx.proxad.net) has joined #ceph
[17:20] * wschulze (~wschulze@cpe-69-206-251-158.nyc.res.rr.com) Quit (Quit: Leaving.)
[17:23] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[17:27] * nwat_ (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[17:28] * imriz (~imriz@82.81.163.130) Quit (Ping timeout: 480 seconds)
[17:33] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) Quit (Quit: Leaving.)
[17:34] * canta (~canta@77-22-176-164-dynip.superkabel.de) Quit (Quit: WeeChat 0.4.3)
[17:37] * shang (~ShangWu@221x241x135x194.ap221.ftth.ucom.ne.jp) has joined #ceph
[17:39] * shang (~ShangWu@221x241x135x194.ap221.ftth.ucom.ne.jp) Quit ()
[17:39] * narb (~Jeff@38.99.52.10) has left #ceph
[17:40] * danieagle (~Daniel@186.214.77.228) has joined #ceph
[17:40] * oblu (~o@62.109.134.112) Quit (Quit: ~)
[17:41] * shang (~ShangWu@221x241x135x194.ap221.ftth.ucom.ne.jp) has joined #ceph
[17:41] * haomaiwang (~haomaiwan@li671-185.members.linode.com) Quit (Ping timeout: 480 seconds)
[17:44] * nwat (~textual@eduroam-249-245.ucsc.edu) has joined #ceph
[17:46] * oblu (~o@62.109.134.112) has joined #ceph
[17:50] * shang (~ShangWu@221x241x135x194.ap221.ftth.ucom.ne.jp) Quit (Quit: Ex-Chat)
[17:53] <ultimape> ceph-deploy mon create is very particular about the host name
[17:53] * dgbaley27 (~matt@c-98-245-167-2.hsd1.co.comcast.net) has joined #ceph
[17:53] <ultimape> or rather, it doesn't fail out when typoing it
[18:06] <ultimape> Woo! rebalancing 192 pgs
[18:07] <ultimape> managed to get a 1 server cluster up afterall.
[18:14] * oblu (~o@62.109.134.112) Quit (Quit: ~)
[18:15] * sarob (~sarob@2601:9:1d00:c7f:d93d:aa27:dd25:f976) has joined #ceph
[18:20] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) has joined #ceph
[18:20] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[18:21] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[18:23] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Read error: Operation timed out)
[18:23] * oblu (~o@62.109.134.112) has joined #ceph
[18:23] * sarob (~sarob@2601:9:1d00:c7f:d93d:aa27:dd25:f976) Quit (Ping timeout: 480 seconds)
[18:23] * zerick (~eocrospom@190.118.43.113) Quit (Ping timeout: 480 seconds)
[18:28] * madkiss (~madkiss@213162068041.public.t-mobile.at) has joined #ceph
[18:31] * madkiss (~madkiss@213162068041.public.t-mobile.at) Quit ()
[18:35] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[18:36] * madkiss1 (~madkiss@178.188.60.118) has joined #ceph
[18:38] * madkiss1 (~madkiss@178.188.60.118) Quit ()
[18:43] * madkiss (~madkiss@178.188.60.118) Quit (Ping timeout: 480 seconds)
[18:44] * m0e (~Moe@41.45.48.221) Quit (Quit: This computer has gone to sleep)
[18:46] * m0e (~Moe@41.45.48.221) has joined #ceph
[18:48] * cookednoodles (~eoin@eoin.clanslots.com) has joined #ceph
[18:50] * nwat (~textual@eduroam-249-245.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[18:51] * nwat (~textual@eduroam-249-245.ucsc.edu) has joined #ceph
[18:52] <ultimape> how long should it take to create a healthy cluster on freshly zapped drives? I would assume that there is next to nothing for them to shunt around and would be quick like a quick-format option
[18:56] <ultimape> this is my ceph -w screen. not sure how to tell what is going on: http://hastebin.com/wozulacuco.sm
[19:11] * angdraug (~angdraug@63.142.161.17) has joined #ceph
[19:14] * cookednoodles (~eoin@eoin.clanslots.com) Quit (Quit: Ex-Chat)
[19:15] * aldavud_ (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[19:15] * sarob (~sarob@2601:9:1d00:c7f:7071:9e67:9485:1f00) has joined #ceph
[19:17] * oblu (~o@62.109.134.112) Quit (Quit: ~)
[19:20] * aldavud_ (~aldavud@212.243.10.250) has joined #ceph
[19:22] <ghartz_> ultimape, what is your replica size ?
[19:22] <ghartz_> you have enough hosts ?
[19:23] * sarob (~sarob@2601:9:1d00:c7f:7071:9e67:9485:1f00) Quit (Ping timeout: 480 seconds)
[19:24] <ultimape> I have osd pool default size = 2 in the ceph.conf
[19:24] <ultimape> how do I query the replica size?
[19:30] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[19:31] <Vacum> ultimape: each pool can have its own replication size. run ceph osd dump and look at the top of the output. each pool will be listed there
[19:32] <Vacum> ultimape: but that alone is not sufficient. the crushrule and your crushmap come into play as well. if you have a replication size of 2, but the pool's crushrule defines "put each replica on a different host", on a single host setup this is not possible and you will see degraded pgs
[19:33] <ultimape> I put in "osd crush chooseleaf type = 0".
[19:34] * danieagle (~Daniel@186.214.77.228) Quit (Quit: Obrigado por Tudo! :-) inte+ :-))
[19:35] * aldavud (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[19:35] * aldavud_ (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[19:36] <Vacum> ultimape: I assume this only defines what is considered a "leaf". but it does not influence at which level in the hierarchy the distinct replicas are chosen from.
[19:37] <Vacum> ultimape: in the crush rule you will see something like step chooseleaf firstn 0 type host
[19:37] <Vacum> ultimape: that says "find as many leafs as needed, but each on a different type host"
[19:37] * KaZeR (~kazer@c-67-161-64-186.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[19:38] <Vacum> I might be wrong though :)
[19:40] <ultimape> I just found crushtool, reading through the decomiled map now.
[19:43] <ultimape> looks like i'll need to read through the rules. I didn't realize this was autogenerated by the cluster - I thought it was pulling from ceph.conf. here's the curren set : http://hastebin.com/ovupomobom.rb
[19:45] * jeremy___s (~jeremy__s@AStDenis-552-1-167-139.w80-8.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[19:45] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[19:45] * aldavud_ (~aldavud@212.243.10.250) has joined #ceph
[19:45] <Vacum> ultimape: if you change line 8 to ... type osd, it should put replicas of the same PGs to the same host, on different drives
[19:46] <Vacum> ultimape: which is what I talked about some hours ago :) if you later add in more hosts, you will end up with some PGs that have both replicas on the same host.
[19:46] * richardk (richardk@b.clients.kiwiirc.com) has joined #ceph
[19:46] <Vacum> ultimape: so once you add in more hosts, you need to change back that rule
[19:47] <ultimape> I thought that "osd crush chooseleaf type = 0" would do that. Turns out I copied it from one of the dumpling docs. damn you google. heh.
[19:47] <Vacum> ultimape: or you instead now create a seond ruleset with a different name, ie replicated_ruleset_onehost and change the ruleset of all pools to that new ruleset. and later change it back to the default one
[19:49] * m0e (~Moe@41.45.48.221) Quit (Quit: This computer has gone to sleep)
[19:51] * primechu_ (~primechuc@173-17-128-36.client.mchsi.com) has joined #ceph
[19:52] * primechuck (~primechuc@173-17-128-36.client.mchsi.com) Quit (Ping timeout: 480 seconds)
[19:54] <richardk> Hey all, I'm trying to find out why a new hardware configuration I've built out is having awful write performance. We've been running Ceph with lots of Dell 1U R610 2 x quad core 32GB RAM nodes with a PERC 6i controller and 1 10K SAS OS drive, 1 Seagate 240GB 600 Pro SSD, and 4 x 10K IBM SAS drives with a lot of success (sequential writes are betwe
[19:54] <richardk> en 200MB/s and 400MB/s depending on load,etc.).
[19:55] <richardk> The SSD is sliced up for each OSD (1 10k IBM SAS drive per OSD)
[19:55] <Sysadmin88> and 200-400MB/s is awful performance?
[19:55] <richardk> no sorry more coming :)
[19:57] <richardk> Recently we've built out a R510 build 2 x quad core 32GB RAM, PERC H700 controller 1 x 15K SAS OS drive, 2 x Seagate 600 SSD, 11 x 4TB WD Black 7200RPM SATA
[19:57] <richardk> with the same ceph.conf settings as the other build, we get between 15 and 30MB/s on average
[19:58] <richardk> I should mention all machines have dual port Intel 10Gb NICs that are bonded, so 20Gb LACP
[19:58] <Sysadmin88> that is much worse
[19:58] <richardk> Yeah definitely
[19:59] <Sysadmin88> so you added that to your fast config and it slowed the whole lot down?
[19:59] <Sysadmin88> or a new cluster?
[19:59] <richardk> We'll see spikes at the beginning of a benchmark that look okay-ish, around 150MB/s, then it will quickly drop off to 30MB/s or lower as the benchmark goes on
[19:59] * m0e (~Moe@41.45.48.221) has joined #ceph
[19:59] <richardk> This is a new pool
[20:00] * gucki (~smuxi@p549F89EA.dip0.t-ipconnect.de) has joined #ceph
[20:00] <gucki> gi guys
[20:00] <gucki> hi guys
[20:00] <Sysadmin88> because that host has a lot of storage compared to your others... might end up with a ton of the data on it
[20:00] <richardk> So in our ceph cluster we have a SAS01 pool and a SATA01 pool
[20:00] <gucki> doesn't ceph-extras have packages for trusty yet?
[20:01] <richardk> Sysadmin88, since it is a separate pool from the SAS build we wouldn't be mixing placement groups or data across the different pools right?
[20:01] <gucki> http://ceph.com/packages/ceph-extras/debian/dists/
[20:02] <richardk> the SAS01 pool still performs great
[20:02] <Sysadmin88> how many of those new sata hosts did you put in?
[20:03] <richardk> 4
[20:03] <richardk> well I have 3 on an internal test cluster and 4 in production
[20:03] <richardk> but those are separate clusters all together
[20:04] <Vacum> richardk: did you do some low level hdd stress tests on the new machines? so see possible raw performance on the ssd for random write i/o for example
[20:05] <ultimape> Vacum, that did it, I added a new ruleset and told the pools to use that one instead.
[20:05] <ultimape> :)
[20:05] <Vacum> ultimape: good :)
[20:05] * madkiss (~madkiss@178.188.60.118) Quit (Read error: Connection reset by peer)
[20:05] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[20:05] <richardk> Vacum: I've done some basic tests on the WD Black SATA 7200RPM drives, sequential writes are pretty much within vendor specs ~90-120MB/s
[20:05] <Vacum> ultimape: later when you add a new machine, change the rule back to the default
[20:06] <ultimape> yeah, I'm giong over my notes now to mention that steop
[20:06] <Vacum> richardk: I would jump on those SSDs first. you are packing 5 OSD journals onto one ssd?
[20:06] <richardk> The same SSDs are used in our SAS deployment and appear to be doing really well, since we get ~300MB/s writes on that and the SAS drives are really only capable of around 150MB/s
[20:07] <Vacum> ultimape: note that changing the rule _will_ shuffle many of your PGs around, resulting in major backfilling and network i/o
[20:07] <richardk> Vacum: Yes 5 OSDs on 1 SSD and 6 on the other SSD in the box (2 SSD per 2U box with 11 OSDs)
[20:08] <ultimape> vacmum, yeah, i'll plan on having something faster in there for the journals and a dedicated switch to deal with the load.
[20:08] <Vacum> richardk: and those are SAS SSDs?
[20:08] <ultimape> *by that point
[20:08] <richardk> Vacum: No, sorry I forgot to mention that, they're SATA SSDs in both deployments
[20:08] <Vacum> ultimape: note, you do NOT want to change IP addresses of your OSD nodes on a cluster.
[20:09] <richardk> Vacum: this is the exact SSD http://www.seagate.com/internal-hard-drives/solid-state-hybrid/600-pro-ssd/?sku=ST240FP0021#specs
[20:10] <Vacum> ultimape: if you are planning on using a dedicated replication network with dedicated network cards and switch(es), do it from day 1. or at least set it up with two networks, even if it uses the same NICs and switches
[20:11] * ajazdzewski (~quassel@2001:4dd0:ff00:9081:5023:360f:55ac:ce71) has joined #ceph
[20:12] <ultimape> does that cause problems otherwise?
[20:12] <ultimape> I dont' have the nics or switches yet.
[20:12] <Vacum> richardk: mh, hard to tell. if that SSD is doing fine in your live setup with 4 OSD journals on it, I don't see a reason why it shouldn#t cope with 5 :)
[20:13] <Vacum> ultimape: you can add two IPs of two networks to the same NIC for now. changing IPs later on your cluster will be hell. well, at least from my experience. perhaps there is a magic switch to do it easy :)
[20:13] * gucki_ (~smuxi@p549F89EA.dip0.t-ipconnect.de) has joined #ceph
[20:13] * gucki_ (~smuxi@p549F89EA.dip0.t-ipconnect.de) Quit (Read error: Connection reset by peer)
[20:13] * gucki_ (~smuxi@p549F89EA.dip0.t-ipconnect.de) has joined #ceph
[20:13] * gucki_ (~smuxi@p549F89EA.dip0.t-ipconnect.de) Quit (Read error: Connection reset by peer)
[20:13] * gucki (~smuxi@p549F89EA.dip0.t-ipconnect.de) Quit (Remote host closed the connection)
[20:13] <Vacum> sorry, afk for a while now
[20:14] <ultimape> thanks for your help!
[20:14] * ultimape is now known as utlimaway
[20:15] * sarob (~sarob@2601:9:1d00:c7f:45a8:b145:fef3:3854) has joined #ceph
[20:20] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) Quit (Read error: Connection reset by peer)
[20:20] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) has joined #ceph
[20:20] <richardk> Vacum: I agree :)
[20:22] <richardk> The main differences between the builds is 10K SAS OSD vs 7.2k SATA OSD and PERC 6i vs PERC H700 controllers
[20:23] * sarob (~sarob@2601:9:1d00:c7f:45a8:b145:fef3:3854) Quit (Ping timeout: 480 seconds)
[20:26] * yuriw (~Adium@AMarseille-151-1-109-233.w92-150.abo.wanadoo.fr) has joined #ceph
[20:26] * vbellur1 (~vijay@122.167.223.88) has joined #ceph
[20:27] * yuriw (~Adium@AMarseille-151-1-109-233.w92-150.abo.wanadoo.fr) has left #ceph
[20:30] * vbellur (~vijay@122.172.194.71) Quit (Ping timeout: 480 seconds)
[20:32] * aldavud_ (~aldavud@212.243.10.250) Quit (Read error: Operation timed out)
[20:34] * aldavud (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[20:39] * ajazdzewski (~quassel@2001:4dd0:ff00:9081:5023:360f:55ac:ce71) Quit (Remote host closed the connection)
[20:40] * rendar (~I@87.19.182.19) Quit (Read error: Operation timed out)
[20:44] * wschulze (~wschulze@cpe-69-206-251-158.nyc.res.rr.com) has joined #ceph
[20:47] * madkiss (~madkiss@178.188.60.118) Quit (Quit: Leaving.)
[20:48] * hitsumabushi is now known as zz_hitsumabushi
[20:48] <richardk> Vacum: For fun I marked a bunch of the OSD down, so that there is only 3 OSD per SSD. Performance was the same, possibly worse.
[20:48] * KevinPerks (~Adium@cpe-174-098-096-200.triad.res.rr.com) has joined #ceph
[20:49] * analbeard (~shw@support.memset.com) has joined #ceph
[20:56] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[20:56] * aldavud_ (~aldavud@212.243.10.250) has joined #ceph
[20:56] * richardk (richardk@b.clients.kiwiirc.com) Quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
[21:12] * aldavud (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[21:12] * aldavud_ (~aldavud@212.243.10.250) Quit (Ping timeout: 480 seconds)
[21:15] * m0e (~Moe@41.45.48.221) Quit (Quit: This computer has gone to sleep)
[21:15] * sarob (~sarob@2601:9:1d00:c7f:f088:e799:140d:f20) has joined #ceph
[21:20] * aldavud (~aldavud@212.243.10.250) has joined #ceph
[21:23] * sarob (~sarob@2601:9:1d00:c7f:f088:e799:140d:f20) Quit (Ping timeout: 480 seconds)
[21:23] * Discovery (~Discovery@193.0.200.73) has joined #ceph
[21:25] * aldavud_ (~aldavud@212.243.10.250) has joined #ceph
[21:37] * Cube (~Cube@66-87-130-64.pools.spcsdns.net) has joined #ceph
[21:48] * davidz (~Adium@ip68-4-173-198.oc.oc.cox.net) Quit (Quit: Leaving.)
[21:55] * sleinen1 (~Adium@2001:620:0:26:ec6f:f7e4:bc8e:8a83) Quit (Quit: Leaving.)
[21:55] * sleinen (~Adium@84-72-160-233.dclient.hispeed.ch) has joined #ceph
[21:56] * sleinen1 (~Adium@2001:620:0:26:a51d:fc03:2abd:dc61) has joined #ceph
[21:57] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[22:03] * sleinen (~Adium@84-72-160-233.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[22:08] * cookednoodles (~eoin@eoin.clanslots.com) has joined #ceph
[22:10] * ghartz_ (~ghartz@ip-68.net-80-236-84.joinville.rev.numericable.fr) Quit (Remote host closed the connection)
[22:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[22:23] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[22:47] * Cube1 (~Cube@66-87-131-149.pools.spcsdns.net) has joined #ceph
[22:54] * Cube (~Cube@66-87-130-64.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[22:57] * midekra_ (~dennis@ariel.xs4all.nl) has joined #ceph
[22:57] * midekra (~dennis@ariel.xs4all.nl) Quit (Ping timeout: 480 seconds)
[23:06] * midekra_ (~dennis@ariel.xs4all.nl) Quit (Read error: Connection reset by peer)
[23:09] * BManojlovic (~steki@cable-94-189-165-169.dynamic.sbb.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[23:15] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) has joined #ceph
[23:18] * Discovery (~Discovery@193.0.200.73) Quit (Ping timeout: 480 seconds)
[23:19] * cookednoodles (~eoin@eoin.clanslots.com) Quit (Quit: Ex-Chat)
[23:22] * rendar (~I@87.19.182.19) has joined #ceph
[23:23] * sarob (~sarob@c-76-102-72-171.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[23:28] * angdraug (~angdraug@63.142.161.17) Quit (Quit: Leaving)
[23:49] * kfei (~root@61-227-52-61.dynamic.hinet.net) Quit (Ping timeout: 480 seconds)
[23:56] * rendar (~I@87.19.182.19) Quit ()

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.