#ceph IRC Log

Index

IRC Log for 2013-05-05

Timestamps are in GMT/BST.

[0:08] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[0:12] * rustam (~rustam@94.15.91.30) has joined #ceph
[0:13] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[0:19] * scalability-junk (uid6422@tooting.irccloud.com) has joined #ceph
[0:30] * rustam (~rustam@94.15.91.30) has joined #ceph
[0:31] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[0:37] * BillK (~BillK@58-7-104-61.dyn.iinet.net.au) has joined #ceph
[0:57] * rustam (~rustam@94.15.91.30) has joined #ceph
[1:13] * tnt (~tnt@109.130.111.54) Quit (Ping timeout: 480 seconds)
[1:46] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[1:58] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[2:48] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[2:57] * sjustlaptop (~sam@71-83-191-116.dhcp.gldl.ca.charter.com) has joined #ceph
[3:03] * TMM (~hp@535240C7.cm-6-3b.dynamic.ziggo.nl) Quit (Quit: Bye)
[3:04] * TMM (~hp@535240C7.cm-6-3b.dynamic.ziggo.nl) has joined #ceph
[3:14] * sjustlaptop (~sam@71-83-191-116.dhcp.gldl.ca.charter.com) Quit (Remote host closed the connection)
[3:18] * rustam (~rustam@94.15.91.30) has joined #ceph
[3:22] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[3:34] * glowell (~glowell@c-98-210-224-250.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[3:37] * glowell (~glowell@c-98-210-224-250.hsd1.ca.comcast.net) has joined #ceph
[4:10] * mnash (~chatzilla@66-194-114-178.static.twtelecom.net) Quit (Quit: ChatZilla 0.9.90 [Firefox 20.0.1/20130409194949])
[4:37] * diegows (~diegows@190.190.2.126) Quit (Ping timeout: 480 seconds)
[4:38] * noahmehl_ (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[4:44] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Ping timeout: 480 seconds)
[4:44] * noahmehl_ is now known as noahmehl
[5:05] * mega_au (~chatzilla@84.244.21.218) has joined #ceph
[5:32] * coyo (~unf@pool-71-164-242-68.dllstx.fios.verizon.net) has joined #ceph
[6:17] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Quit: noahmehl)
[6:27] * rustam (~rustam@94.15.91.30) has joined #ceph
[6:31] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[6:40] * gregaf1 (~Adium@cpe-76-174-249-52.socal.res.rr.com) has joined #ceph
[6:44] * gregaf1 (~Adium@cpe-76-174-249-52.socal.res.rr.com) Quit ()
[7:01] * vipr_ (~vipr@78-23-112-42.access.telenet.be) has joined #ceph
[7:08] * vipr (~vipr@78-23-113-70.access.telenet.be) Quit (Ping timeout: 480 seconds)
[7:50] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) has joined #ceph
[7:50] * ChanServ sets mode +o scuttlemonkey
[8:05] * itamar_ (~itamar@82.166.185.149) has joined #ceph
[8:17] * musca (musca@tyrael.eu) Quit (Quit: ZNC - http://znc.sourceforge.net)
[8:18] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[8:19] * KindOne (KindOne@0001a7db.user.oftc.net) has joined #ceph
[8:29] * leo (~leo@27.106.31.36) has joined #ceph
[8:41] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) Quit (Ping timeout: 480 seconds)
[8:44] * coyo (~unf@00017955.user.oftc.net) Quit (Quit: F*ck you, I'm a daemon.)
[8:53] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[8:53] * KindOne (KindOne@0001a7db.user.oftc.net) has joined #ceph
[8:54] * musca (musca@tyrael.eu) has joined #ceph
[8:59] * leo (~leo@27.106.31.36) Quit (Read error: No route to host)
[9:06] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) has joined #ceph
[9:06] * ChanServ sets mode +o scuttlemonkey
[9:09] * rustam (~rustam@94.15.91.30) has joined #ceph
[9:11] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[9:16] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) Quit (Ping timeout: 480 seconds)
[9:40] * leo (~leo@27.106.31.36) has joined #ceph
[10:04] * vipr (~vipr@78-23-113-244.access.telenet.be) has joined #ceph
[10:05] * leo (~leo@27.106.31.36) Quit (Read error: No route to host)
[10:11] * vipr_ (~vipr@78-23-112-42.access.telenet.be) Quit (Ping timeout: 480 seconds)
[10:13] * dragonfly (~root@118.249.168.209) has joined #ceph
[10:14] * dragonfly (~root@118.249.168.209) Quit ()
[10:38] * tnt (~tnt@109.130.111.54) has joined #ceph
[10:54] * vo1d (~v0@212-183-100-112.adsl.highway.telekom.at) has joined #ceph
[11:01] * v0id (~v0@194.118.210.190) Quit (Ping timeout: 480 seconds)
[11:38] * rustam (~rustam@94.15.91.30) has joined #ceph
[11:59] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) has joined #ceph
[12:07] * loicd (~loic@magenta.dachary.org) has joined #ceph
[12:08] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[12:14] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[12:33] * DarkAceZ (~BillyMays@50.107.54.92) Quit (Ping timeout: 480 seconds)
[12:35] * musca (musca@tyrael.eu) Quit (Remote host closed the connection)
[12:42] * musca (musca@tyrael.eu) has joined #ceph
[12:49] * s_yangbin (~oftc-webi@159.226.169.127) has joined #ceph
[12:52] * DarkAceZ (~BillyMays@50.107.54.92) has joined #ceph
[13:07] <s_yangbin> hi,guys~,sorry in advance about my bad english, is radosgw support CORS?
[13:26] <tnt> s_yangbin: radosgw is a fastcgi process so you always have a http front end in front of it and you can tell this frontend (apache/lighttpd/nginx/...) to reply to the OPTION request with the appropriate Access-Control-Allow-Origin:
[13:29] <s_yangbin> thanks for your reply tnt, I new to this, is there any documents for this?
[13:33] <tnt> not that I know. And probably not in radosgw documentation because it really has nothing to do with radogw, it's handled by the upper http frontend layer
[13:36] <s_yangbin> I see. thank you very much~
[13:37] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[13:38] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) Quit (Quit: Leaving.)
[13:45] * s_yangbin (~oftc-webi@159.226.169.127) Quit (Quit: Page closed)
[13:57] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[13:58] * rustam (~rustam@94.15.91.30) has joined #ceph
[14:00] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[14:02] * diegows (~diegows@190.190.2.126) has joined #ceph
[14:06] * rustam (~rustam@94.15.91.30) has joined #ceph
[14:07] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[14:08] * leseb (~Adium@82.226.32.84) has joined #ceph
[14:19] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Read error: Connection reset by peer)
[14:21] * sleinen (~Adium@2001:620:0:25:a9ae:dc2f:fbc1:224e) has joined #ceph
[14:29] * leseb (~Adium@82.226.32.84) Quit (Quit: Leaving.)
[14:46] * rustam (~rustam@94.15.91.30) has joined #ceph
[14:47] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[14:49] * rustam (~rustam@94.15.91.30) has joined #ceph
[14:50] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[15:18] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) has joined #ceph
[15:19] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) Quit ()
[15:19] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) has joined #ceph
[15:25] * rustam (~rustam@94.15.91.30) has joined #ceph
[15:27] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[15:27] * itamar_ (~itamar@82.166.185.149) Quit (Quit: Leaving)
[15:30] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[15:30] * loicd (~loic@magenta.dachary.org) has joined #ceph
[15:51] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) has joined #ceph
[15:51] * ChanServ sets mode +o scuttlemonkey
[16:06] * rustam (~rustam@94.15.91.30) has joined #ceph
[16:11] * leseb1 (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) has joined #ceph
[16:11] * leseb (~Adium@pha75-6-82-226-32-84.fbx.proxad.net) Quit (Read error: Connection reset by peer)
[16:26] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[16:39] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) Quit (Ping timeout: 480 seconds)
[16:54] * rustam (~rustam@94.15.91.30) has joined #ceph
[16:55] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[16:57] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[17:04] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[17:05] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) has joined #ceph
[17:05] * ChanServ sets mode +o scuttlemonkey
[17:07] * loicd (~loic@magenta.dachary.org) has joined #ceph
[17:07] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[17:14] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) Quit (Ping timeout: 480 seconds)
[17:16] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[17:31] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:7c46:5e17:1255:444a) Quit (Quit: Leaving.)
[17:45] * jackhill_ (~jackhill@71.20.247.147) Quit (Remote host closed the connection)
[17:46] * diegows (~diegows@190.190.2.126) Quit (Ping timeout: 480 seconds)
[17:52] * derRichard (~derRichar@pippin.sigma-star.at) has joined #ceph
[17:52] <derRichard> hi
[17:53] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) has joined #ceph
[17:53] * ChanServ sets mode +o scuttlemonkey
[18:08] * BillK (~BillK@58-7-104-61.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[18:09] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) Quit (Ping timeout: 480 seconds)
[18:18] <jksM> derRichard, hi
[18:24] <derRichard> i'm new to ceph. i'd like to use it as storage system for kvm. does ceph work well with a small cluster size? 3-4 nodes?
[18:44] <jksM> as far as I know, yes
[18:48] <derRichard> how are you using ceph?
[18:49] * danieagle (~Daniel@186.214.61.67) has joined #ceph
[18:50] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) has joined #ceph
[18:50] * ChanServ sets mode +o scuttlemonkey
[18:50] <vipr> I started with a 3 node cluster (3 OSDs each), and it was very stable
[18:53] <derRichard> i'm wondering how ceph compares to sheepdog. has one of you tested both?
[18:59] * madkiss (~madkiss@2001:6f8:12c3:f00f:4925:7924:6eb2:dc55) has joined #ceph
[19:01] * scuttlemonkey (~scuttlemo@74-130-236-21.dhcp.insightbb.com) Quit (Ping timeout: 480 seconds)
[19:04] <jksM> derRichard, I'm have a 3 node Ceph cluster that stores KVM guests... working very well for me too
[19:05] <derRichard> what happens if one node dies? what is your replication level?
[19:05] <jksM> I'm using replication level 2 for most stuff
[19:06] <jksM> If a node dies, it is noticed by other servers after usually half a minute or so
[19:06] <derRichard> but all kvm guests which were served by that node die too?
[19:06] <jksM> After a short while, the remaining servers starts to replicate the data that was on the failed server, so that the number of replicas get to 2 again
[19:07] <jksM> Oh, I don't run KVM guests on the Ceph servers
[19:07] <jksM> But if you did, they would obviously die (unless you have some special magic sauce)... how would you avoid that?
[19:08] <derRichard> e.g. using a multihomed ip...
[19:08] <jksM> that wouldn't help in itself
[19:09] <jksM> are we talking about the same thing here?
[19:09] <jksM> are you talking about KVM guests running on the server that died (I was)
[19:09] <jksM> or are you talking about KVM guests running on other servers - but have their data stored on the Ceph cluster?
[19:10] <derRichard> the latter :)
[19:10] <jksM> ah, okay - then nothing happens to them
[19:10] <jksM> they just keep on running
[19:11] <derRichard> good to know
[19:11] <jksM> just yesterday I rebooted all my Ceph servers (software upgrades) without causing any downtime for the KVM guests
[19:12] <jksM> obviously had to reboot one server at a time and not all at once :-)
[19:12] <derRichard> haha, sure :)
[19:12] <derRichard> so, you are using qemu's rbd driver?
[19:12] <jksM> yes
[19:12] <derRichard> okay. the qemu manpage never told me about rbd :(
[19:13] <jmlowe> depending on your crushmap and replication level you can do more than one at a time, I did 2 at a time yesterday when I did my upgrades
[19:13] <jksM> it's a bit difficult with a 3 node cluster :)
[19:14] <derRichard> if you have 3 nodes and reboot one, how does ceph know that this is a "regular" reboot and there is no need to start replicating data?
[19:14] <jksM> derRichard, you need to tell it beforehand
[19:14] <jmlowe> 4 nodes 2 racks 2 replicas, crush splits between racks for me
[19:14] <jksM> jmlowe, very neat!
[19:14] <derRichard> nice
[19:14] <jmlowe> derRichard: there is a timeout, also you can 'ceph osd set noout'
[19:15] <derRichard> on each node you use xfs as filesystem? is you filesystem ontop of some raid1/5?
[19:15] <derRichard> (to avoid donetime due to a disk failure)
[19:15] <derRichard> *downtime
[19:15] <jksM> I think the general recommendation is not to use RAID
[19:15] <jmlowe> being marked "out" changes the computation on where the objects should be and thus triggers the cluster to fill in the missing objects "backfill"
[19:15] * musca (musca@tyrael.eu) has left #ceph
[19:15] <jksM> I use btrfs as I'm living on the edge... but I think xfs is the general recommendation
[19:16] <derRichard> jksM: raid is not good because of the overhead?
[19:16] <jmlowe> I use xfs on 4x1tb and 4x3tb hardware raid
[19:16] <jmlowe> with battery backed cache and nobarrier
[19:16] <jksM> derRichard, I'm told that it could have performance implications if you use mdraid for example
[19:17] <jksM> (beyond what you would expect from mdraid itself)
[19:17] * rustam (~rustam@94.15.91.30) has joined #ceph
[19:17] <jmlowe> yeah I wasn't as happy using btrfs raid 10 as I am with hardware raid5
[19:18] <jksM> derRichard, if you're going to test ceph, I would recommend trying it with the qemu from their git repository, as it includes new code that makes it perform a lot better (on my system at least)
[19:18] <derRichard> so, instead of having raid5 on four disks you'd mount all four disks to /a /b /c and /d and run four osds?
[19:18] <jksM> derRichard, exactly
[19:18] <derRichard> jksM: oh, their qemu driver is not mainline? hmm
[19:19] <jmlowe> it is
[19:19] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[19:19] <derRichard> ok, got it
[19:19] <jksM> derRichard, it is, but the newer code has a new feature that makes it perform better
[19:19] <jmlowe> there are some flush with caching fixes
[19:19] <jmlowe> I find the ubuntu 12.10 and 13.04 versions acceptable, ymmv
[19:20] <jksM> the new code makes it possible for qemu to flush the rbd asynchronously
[19:20] <derRichard> thanks a lot for your help! i have to go now.
[19:20] <jksM> I'm not sure, but I think it impacts even if you're running without the cache
[19:20] <jmlowe> I'm not clear on the details myself
[19:21] <jksM> at least it seems that way on my own system
[19:21] <jmlowe> I'll probably wait for ubuntu 13.10
[19:21] <jksM> before when I ran something very write intensive inside a guest, I would see the guest becoming very non-responsive... for example ping time would sky rocket
[19:21] <jksM> with the patch everything is like you would expect it
[19:21] <jmlowe> I'm going to need to schedule 70 or so vm reboots between now and June 4th to get up to qemu 1.4
[19:22] <jksM> writes aren't performed faster ofcourse, but it just "feels faster" :)
[19:22] <jksM> hehe
[19:22] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Remote host closed the connection)
[19:22] <jksM> I migrated my vms around and cleared a server to upgrade, and then migrated back after some testing
[19:22] <jmlowe> I saw some allusion to virtio-block not calling flush, know anything about that?
[19:23] <jksM> so yeah, upgrading qemu is a bit of a pain
[19:23] <jksM> jmlowe, nope, haven't heard about that
[19:23] <jmlowe> I can't seem to live migrate from qemu 1.2 to 1.4
[19:23] <jksM> ah, I wasn't on 1.2
[19:23] <jksM> 1.4 doesn't include the async fix... so I was upgrading from 1.4 to the newest git code
[19:24] <jmlowe> also I had cach on in ceph.conf and not cach=writeback for qemu which is bad afaik
[19:25] <jmlowe> I'd have to reboot to get the guest kernel to pick up the proper device flags
[19:25] <jksM> hmm, I see something about virtio not sending flushes if the kernel version is older than 2.6.32 (inside the guest)
[19:25] <jmlowe> can I ask you to describe your setup and use case in more detail?
[19:26] <jksM> perhaps that was what you were talking about?
[19:26] <jmlowe> yeah, that
[19:26] <jksM> sure, what do you want to know about my setup?
[19:26] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[19:34] <jmlowe> I've got 8 vm hosts and 4 osd hosts split between two data centers that are 50 miles apart with a single uninterrupted piece of fiber connecting them at 40Gbs (4x10Gbs trunking), all my host networking is 10GigE, I have 18 osds 74793 GB / 82874 GB avail, I primarily host vm's for people affiliated with xsede.org wanting to run services, I also do various hpc related support system vms for my employer Indiana University
[19:35] <jksM> ouch, really nice setup! :-)
[19:35] <jmlowe> I have mostly centos 5.9 and centos 6.4 guests, a couple of ubuntu 12.04 and one debian squeeze guests
[19:36] <jmlowe> 79 vm's currently running
[19:37] <jksM> I've got 3 vm hosts and 3 osd hosts in the same data center. Each osd has 3 x 1 Gbps connectivity and the vm hosts just have 1 Gbps. 12 osds in total with 22 TB data available.
[19:38] <jmlowe> I can get about 80MB/s write inside of a vm, how about you?
[19:38] <jksM> I have been testing ceph for about half a year, so it has just been running some test vms to see how Ceph works. After testing the new async flush in qemu, I decided to put my first production vm in place just today! :-)
[19:38] <jmlowe> you and I started banging around on ceph about the same time
[19:40] <jksM> I run a one-man company where I consult and develop various software packages that I also host for the users. My plan was to be able to move from VMs on disks in each server to a Ceph backed system
[19:41] <jksM> My guests are more or less all CentOS 5 and CentOS 6... with one or two Windows Server installations thrown in for added evilness
[19:41] <jksM> I'm not even close to 80 MB/s :-)
[19:41] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) Quit (Ping timeout: 480 seconds)
[19:41] <jksM> My next task for Ceph is actually to find out how to do more precise benchmarking to figure out what to do to improve performance
[19:42] <jksM> But simple tests like doing a dd if=/dev/zero inside a guest gives me about 12 MB/s
[19:42] <jmlowe> I'm really interested to see what cuttlefish brings
[19:42] <jmlowe> I hear good things about performance
[19:43] <jmlowe> I'm always pushing up against my backup window
[19:43] <jksM> Yeah, I have heard that as well... but not too keen on upgrading before it is stable.. had some rough patches even with bobtail ;-)
[19:43] <jksM> I have just noticed that it is now possible to get 10 gbit equipment substantially cheaper than 6 months ago where I bought the current hardware I'm using
[19:44] <jmlowe> I had a pretty rough time with btrfs, lost a lot of data
[19:44] <jksM> Hmm, btrfs hasn't caused me problems at all actually
[19:44] <jksM> But I had problems with osds crashing and not being able to get back up, etc.
[19:44] <jmlowe> I found the bug that caused sparse files on btrfs to get truncated
[19:45] <jksM> but those bugs were fixed, and I chose a lower setting for the target transaction size... and since then everything has been rock stable
[19:45] <jksM> ouch, that was a bad one :-|
[19:45] * gregaf1 (~Adium@2607:f298:a:607:10e3:a393:f44c:3d3) has joined #ceph
[19:45] <jksM> which kernel version do you need to avoid that one?
[19:46] <jksM> I'm wondering where to put my (limited) resources in order to achieve better performance... 10-gbit will set me back about 3500$ with the new low cost equipment
[19:47] <jksM> but I'm not really seeing that much network bandwidth being used - although I wonder if the lower latency in itself would give an improvement
[19:48] <jksM> I'm pretty sure that the lowest-hanging fruit is adding SSD journals, but that probably won't help me read performance
[19:49] * BillK (~BillK@58-7-104-61.dyn.iinet.net.au) has joined #ceph
[19:50] <jmlowe> 3.8 is a minimum
[19:51] <jksM> okay, I got that covered :)
[19:51] <jmlowe> I'm wondering about the ssd block cache that is making waves, I'm thinking some version of that hit the mainline kernel in 3.9
[19:52] <jksM> you mean the ssd block cache that goes on the vm hosts?
[19:52] <jmlowe> I would think separate journals would help your read performance, they can't read when writing so you could squeeze out some more reads while you would normally be writing to the journal
[19:52] * gregaf (~Adium@2607:f298:a:607:3536:c4f2:afcb:c49e) Quit (Ping timeout: 480 seconds)
[19:53] <jmlowe> ssd block cache on the osd's, like a giant raid controller cache
[19:53] <jmlowe> http://bcache.evilpiepirate.org/
[19:53] <jksM> oh! can that ssd be the same ssd as the one the journals are on, or it needs to be seperate?
[19:54] <jksM> thanks for the link!
[19:55] <jmlowe> ok, the caching they added in 3.9 is part of device mapper
[19:55] <jksM> if the cache ssd dies, that would mean loosing the data on all the spindles as well, right?
[19:55] <jmlowe> dm-cache
[19:55] <jksM> (i.e. they could be suffering from missing writeS)
[19:56] <jmlowe> I would think so unless you used writethrough
[19:56] <jksM> I'm using IBM hardware and pretty annoyed at their SSD options (and pricing) :-|
[19:57] <jmlowe> the question is how common is that failure mode, entire disk dies vs slowed or going read only
[19:57] <jksM> I hope they will catch up to the rest of the market soon
[19:57] <jksM> yeah, that's right... in theory ssds should simply go read only
[19:57] <jksM> but in practice I have seen a certain percentage of drives simply going bust out of the blue
[19:57] <jmlowe> I think they use slc exclusively, so that would mean lower densities higher costs lower failure rate
[19:58] <jksM> yep, that's right... but still it is very very costly... I wish they would expand to allow for mlc for less demanding users
[19:58] <jksM> they do offer MLC SSDs right now... but it's a 64 GB drive
[19:59] <jksM> (priced at 500$+)
[19:59] <jmlowe> if I wanted a crappy 64G mlc I'd just stuff a usb key in there
[19:59] <jksM> hehe
[20:00] <jksM> the SLC options are quite costly for a small company like my own
[20:00] <jksM> i.e. cheapest drive is 4000$
[20:00] <jmlowe> ouch
[20:00] <jmlowe> you can get a massive pcie ssd for that
[20:00] <jksM> and that's a 100 GB drive
[20:00] <jmlowe> much
[20:01] <jksM> IBM also sells a "high iops slc" drive... at 33,000$ ;-)
[20:02] <jmlowe> nhm is the guy to ask about what to buy next, I don't have enough data points to make a guess
[20:03] <jksM> okay :-) I'm planning to buy some SuperMicro hardware where I can put ordinary "third party" SSDs in
[20:03] <jksM> I was actually thinking about purchasing one of those new low cost 10 Gbps switches and a bunch or cheap MLC SSDs to use as osds
[20:04] <jksM> and then place the journal on the osds I think
[20:04] <jmlowe> which distro do you use for your vm hosts?
[20:04] <jksM> I haven't fully decided... I was planning to use CentOS as I do that normally
[20:04] <jksM> but for testing the new qemu I have been using Fedora simply as it was easier to get the new qemu running
[20:06] <jmlowe> I was thinking it might be worth it if enough of us would benefit and participate to have some repos for qemu and libvirt with an eye to keeping up with ceph
[20:06] <jksM> yes, that would be really, really nice!
[20:07] <jmlowe> ubuntu has the launchpad ppa thingy in place, yum repos are pretty easy to host
[20:08] <jmlowe> right now I'd like to go to raring but there aren't any 0.56.6 packages built, 0.56.4 is part of the distro
[20:09] <jmlowe> I'd also love to get my hands on that qemu patch and get libvirt 1.0.4 instead of 1.0.2 that comes with the distro
[20:10] <jmlowe> the q35 machine type is broken for libvirt 1.0.2, calls the pcie bus pci and qemu wont' start
[20:10] <jksM> ah okay - I don't use libvirt luckily ;-)
[20:10] <jmlowe> blessing/curse
[20:11] <jksM> yep, like all good stuff :)
[20:12] <jksM> I put my own small system together where I have a small web interface with provisioning, HTML5 VNC for management, etc.
[20:12] <jksM> and then just use qemu directly to boot up vms
[20:13] <jksM> the qemu patch I use is this one: http://git.qemu.org/?p=qemu.git;a=commitdiff;h=dc7588c1eb3008bda53dde1d6b890cd299758155
[20:13] <jmlowe> I do mine by hand with libvirt helping out
[20:14] <jksM> I tried applying it to 1.4.1 by altering the function prototype for bdrv_open to fit the older version, but sadly it wasn't enough to make it work
[20:14] <mrjack> jksM: can you migrate a host from qemu 1.4.1 to git without downtime for the guest?
[20:15] <jksM> mrjack, I think so, yes - but I cannot promise that it works for everyone or in all cases
[20:15] <jksM> (and obviously depends on the git version you're migrating to) ;-)
[20:16] <jksM> I was interested in getting it working on 1.4.1, but I segfaults inside librbd when using the patched rbd driver
[20:16] <jmlowe> hmm, I'd prefer a clean and simple backport to a numbered version to running something from the trunk
[20:16] <jksM> gdb'ing qemu isn't easy it seems.... I couldn't figure out a way to make it attach to the right child process so that I could get a back trace
[20:16] <jksM> jmlowe, exactly
[20:17] <jksM> I need to attach to the child of the first fork and the parent of the second fork... didn't spend the necessary time to work that out
[20:17] <jmlowe> hmm, change log doesn't exactly cover the delta between 1.4.0 and 1.4.1
[20:18] <mrjack> is there a working patch for 1.4.1?
[20:18] <jksM> jmlowe, http://lists.nongnu.org/archive/html/qemu-devel/2013-04/msg02956.html
[20:19] <jksM> mrjack, not that I know of?
[20:42] * BillK (~BillK@58-7-104-61.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[21:12] * lofejndif (~lsqavnbok@tor01.spacedump.net) has joined #ceph
[21:31] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:fc26:5536:c014:cb73) has joined #ceph
[21:37] * madkiss (~madkiss@2001:6f8:12c3:f00f:4925:7924:6eb2:dc55) Quit (Ping timeout: 480 seconds)
[21:40] * danieagle (~Daniel@186.214.61.67) Quit (Quit: Inte+ :-) e Muito Obrigado Por Tudo!!! ^^)
[21:59] * wca_ (~will@198.105.209.36) Quit (Quit: leaving)
[22:15] * jcsp (~john@82-71-55-202.dsl.in-addr.zen.co.uk) has joined #ceph
[22:19] * rustam (~rustam@94.15.91.30) has joined #ceph
[22:22] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[22:28] * rustam (~rustam@94.15.91.30) has joined #ceph
[22:29] * jpieper (~josh@209-6-205-161.c3-0.smr-ubr2.sbo-smr.ma.cable.rcn.com) Quit (Remote host closed the connection)
[22:30] * rustam (~rustam@94.15.91.30) Quit (Remote host closed the connection)
[22:47] * madkiss (~madkiss@2001:6f8:12c3:f00f:ed55:caf9:6392:b226) has joined #ceph
[22:51] * mega_au (~chatzilla@84.244.21.218) Quit (Ping timeout: 480 seconds)
[22:52] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:fc26:5536:c014:cb73) Quit (Ping timeout: 480 seconds)
[23:01] * madkiss (~madkiss@2001:6f8:12c3:f00f:ed55:caf9:6392:b226) Quit (Quit: Leaving.)
[23:20] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[23:21] * tkensiski (~tkensiski@c-98-234-160-131.hsd1.ca.comcast.net) has joined #ceph
[23:21] * tkensiski (~tkensiski@c-98-234-160-131.hsd1.ca.comcast.net) has left #ceph
[23:33] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) has joined #ceph
[23:47] <leseb1> joshd: are you around?
[23:52] * sleinen (~Adium@2001:620:0:25:a9ae:dc2f:fbc1:224e) Quit (Quit: Leaving.)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.