#ceph IRC Log


IRC Log for 2010-12-16

Timestamps are in GMT/BST.

[0:39] * gregorg_taf (~Greg@ Quit (Ping timeout: 480 seconds)
[0:40] * gregorg_taf (~Greg@ has joined #ceph
[1:44] * Meths_ (rift@ has joined #ceph
[1:51] * Meths (rift@ Quit (Ping timeout: 480 seconds)
[1:52] * Meths_ is now known as Meths
[2:10] * cmccabe (~cmccabe@dsl081-243-128.sfo1.dsl.speakeasy.net) Quit (Quit: Leaving.)
[3:01] * joshd (~joshd@ip-66-33-206-8.dreamhost.com) Quit (Quit: Leaving.)
[3:30] * bchrisman (~Adium@70-35-37-146.static.wiline.com) Quit (Quit: Leaving.)
[4:25] * alexxy[home] (~alexxy@ has joined #ceph
[4:25] * alexxy (~alexxy@ Quit (Read error: Connection reset by peer)
[5:05] * greglap (~Adium@ has joined #ceph
[5:41] * bchrisman (~Adium@c-24-130-226-22.hsd1.ca.comcast.net) has joined #ceph
[5:59] * greglap (~Adium@ Quit (Read error: Connection reset by peer)
[6:07] * MarkN (~nathan@mail.zomojo.com) has left #ceph
[6:10] * MarkN (~nathan@mail.zomojo.com) has joined #ceph
[6:10] * MarkN (~nathan@mail.zomojo.com) has left #ceph
[6:15] * greglap1 (~Adium@cpe-76-90-74-194.socal.res.rr.com) has joined #ceph
[6:49] * ijuz_ (~ijuz@p4FFF5C95.dip.t-dialin.net) has joined #ceph
[6:57] * ijuz__ (~ijuz@p4FFF652C.dip.t-dialin.net) Quit (Ping timeout: 480 seconds)
[6:57] * f4m8_ is now known as f4m8
[8:48] * allsystemsarego (~allsystem@ has joined #ceph
[8:49] * bchrisman (~Adium@c-24-130-226-22.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[9:49] <wido> jantje_: Are you there?
[9:50] <wido> I'm trying to get my journals running on a tmpfs (/var/run), but I keep getting "unable to open journal /var/run/ceph/osd0-journal: error 22: Invalid argument"
[9:50] <wido> journal size is specified and OSD has write permission in the directories
[9:50] <wido> from a strace: open("/var/run/ceph/osd0-journal", O_RDWR|O_CREAT|O_SYNC|O_DIRECT, 0644) = -1 EINVAL (Invalid argument)
[10:04] * verwilst (~verwilst@router.begen1.office.netnoc.eu) has joined #ceph
[11:06] <wido> Well, after some searching I found you have to place 'journal dio = false' in your ceph.conf (I've put it in the OSD section)
[11:06] <wido> Just writing it down here so it gets in the IRC logs
[11:06] <wido> If you want your OSD journal on tmpfs, use journal dio = false
[11:06] <wido> Found it in: http://tracker.newdream.net/issues/336
[12:06] * sentinel_e86_ (~sentinel_@ has joined #ceph
[12:07] * sentinel_e86 (~sentinel_@ Quit (Quit: sh** happened)
[13:07] * Yoric (~David@ has joined #ceph
[14:35] * f4m8 is now known as Guest1346
[14:39] * O01 (~2@82VAAAWFF.tor-irc.dnsbl.oftc.net) has joined #ceph
[14:44] * O01 (~2@82VAAAWFF.tor-irc.dnsbl.oftc.net) has left #ceph
[15:30] * Guest1346 is now known as f4m8
[16:14] * allsystemsarego (~allsystem@ Quit (Quit: Leaving)
[17:06] * bchrisman (~Adium@c-24-130-226-22.hsd1.ca.comcast.net) has joined #ceph
[17:18] * Yoric_ (~David@ has joined #ceph
[17:18] * Yoric (~David@ Quit (Read error: Connection reset by peer)
[17:18] * Yoric_ is now known as Yoric
[17:33] * greglap1 (~Adium@cpe-76-90-74-194.socal.res.rr.com) Quit (Quit: Leaving.)
[17:36] <sage> wido: can you include the commit id for the kernel you're running, and a copy/paste of the warnings you just saw for that bug?
[17:42] <wido> sage: you mean the kernel I was running yesterday?
[17:43] <wido> the commit id of my current kernel is in the update from today
[17:43] <wido> seems to be two different issues?
[17:49] * bchrisman (~Adium@c-24-130-226-22.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[17:51] * greglap (~Adium@ has joined #ceph
[17:56] <wido> greglap: It was my SSD causing the slowdown
[17:56] <wido> placed the journals on tmpfs and everything went fine
[17:56] <greglap> wido: this was with the rsync?
[17:56] <wido> yes
[17:56] <greglap> it started okay and then crapped out after a few minutes?
[17:56] <wido> 196k files, 74GB of data
[17:57] <wido> no, the SSD was too slow it seems, couldn't keep up, load of the machine went up and up
[17:57] <greglap> ah, right
[17:57] <wido> with the OSD journal on tmpfs the load stayed somewhere around 2 and rsync finished succesfully after a few hours
[17:58] <greglap> do you have any points of comparison for rsyncing it on not-ceph?
[17:58] <wido> No, but that's something i'll test.
[17:59] <wido> Now I know what my bottleneck was, I can do such a test
[17:59] * verwilst (~verwilst@router.begen1.office.netnoc.eu) Quit (Quit: Ex-Chat)
[17:59] <greglap> it's nothing urgent, I just don't have enough points of comparison myself to know if "a few hours" is slow or not :)
[17:59] <wido> I'm not sure either, since it were a lot of small files, which always cause a lot of overhead
[17:59] <greglap> I thought maybe if they were backups that you rsynced them around to other servers too :D
[18:00] <wido> It are backups I got from my backup server :)
[18:00] <wido> I'm afk for dinner
[18:00] <greglap> k
[18:23] * greglap (~Adium@ Quit (Quit: Leaving.)
[18:49] * joshd (~joshd@ip-66-33-206-8.dreamhost.com) has joined #ceph
[18:51] * cmccabe (~cmccabe@adsl-76-202-117-54.dsl.pltn13.sbcglobal.net) has joined #ceph
[18:56] * bchrisman (~Adium@70-35-37-146.static.wiline.com) has joined #ceph
[18:59] * alexxy (~alexxy@ has joined #ceph
[19:00] * Yoric (~David@ Quit (Quit: Yoric)
[19:03] * alexxy[home] (~alexxy@ Quit (Ping timeout: 480 seconds)
[19:12] * sagelap (~sage@ has joined #ceph
[19:22] * NoahWatkins (~NoahWatki@soenat3.cse.ucsc.edu) has joined #ceph
[20:19] * Meths_ (rift@ has joined #ceph
[20:23] * Meths (rift@ Quit (Read error: Operation timed out)
[20:24] * Meths_ is now known as Meths
[20:52] <wido> gregorg_taf: you still there?
[20:58] <gregaf> wido: oh, somebody knows who that is? I haven't seen him talk
[20:58] <gregaf> (he's not me, but I'm heading out for lunch now, be back in 30 or so)
[21:00] <wido> gregaf: ok :P
[21:00] <wido> I was looking for you, see you in 30
[21:21] <wido> gregaf: I'm going afk, just wanted to let you know that I'll give the rsync a benchmark this night
[21:21] <wido> removing the 196k files (74GB) right now, but that rm is already running for 1.5 hours
[21:21] <wido> i'll post my rsync times in your issue
[21:21] <wido> ttyl!
[21:43] * jantje (~jan@paranoid.nl) has joined #ceph
[21:48] <gregaf> wido: thanks!
[21:49] * jantje_ (~jan@paranoid.nl) Quit (Ping timeout: 480 seconds)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.