#ceph IRC Log

Index

IRC Log for 2012-06-02

Timestamps are in GMT/BST.

[0:00] <dmick> I think they compete for them, yes
[0:00] <elder> OK.
[0:00] <dmick> not sure what concurrency it's using
[0:02] <elder> Looks like it took about an hour. My machine took about 12 minutes.
[0:03] <Tv_> elder: it gets 16 vcpus
[0:03] <dmick> looks like it does -j 16
[0:03] <dmick> (jinx)
[0:04] <elder> And now I have to wait for it to upload somewhere I guess...
[0:04] <elder> There we go. Testing.
[0:04] <Tv_> heh, just when i log in to look at it, it goes idle ;)
[0:04] <dmick> yeah
[0:05] * s[X]_ (~sX]@ppp59-167-154-113.static.internode.on.net) has joined #ceph
[0:08] * lofejndif (~lsqavnbok@83TAAGF36.tor-irc.dnsbl.oftc.net) has joined #ceph
[0:14] <elder> dmick, is it possible we have that old problem with New! grub not selecting the right kernel image again, now that we're running with the new Ubuntu?
[0:16] <elder> Well, maybe not. But I just rebooted and the machines didn't load my new image. Still looking.
[0:17] <elder> It does look a bit like that though.
[0:27] * yanzheng (~zhyan@101.84.4.69) has joined #ceph
[0:45] <sagewk> sjust: pushed wip-pgls
[1:00] * adjohn (~adjohn@50-0-133-101.dsl.static.sonic.net) has joined #ceph
[1:01] * bchrisman (~Adium@108.60.121.114) Quit (Quit: Leaving.)
[1:09] <sagewk> anyone who wants to sanity check wip-pgls, now's your chance
[1:09] <sagewk> gregaf: ^
[1:10] <gregaf> looks not-insane to me
[1:11] <sjust> hang on
[1:19] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[1:23] * josef (~seven@nat-pool-rdu.redhat.com) has joined #ceph
[1:23] <josef> sage: not sure which one of you is on, but i think those spurious writes are the inode_cache
[1:24] <gregaf> sagewk: sjust: nhm: ^
[1:24] <josef> may even bee the space cache too
[1:25] <darkfader> maybe you can see it jumping between positions in slabtop at the time?
[1:27] <nhm> josef: heya!
[1:28] <nhm> josef: Are you looking at the stuff that sage produced or the stuff I showed you the other day?
[1:28] <sagewk> the one i generated yesterday...
[1:28] <sagewk> had lots of scattered writes all over the place
[1:29] <josef> does mount say that inode_cache is enabled?
[1:29] <nhm> sagewk: Yeah, let me check, I can't rememer if that was fake or gen.
[1:29] <josef> if not it may be the space_cache but i doubt it
[1:29] <josef> try nospace_cache just for fun
[1:30] <josef> doesnt look like inode_cache is on by default
[1:30] <josef> there goes that idea
[1:30] <josef> oh well liam is geting into things he shouldnt
[1:31] <josef> i'll look at it for real monday :)
[1:31] <josef> fix one ceph problem and you guys find another one
[1:31] <josef> ;)
[1:31] <nhm> josef: :D
[1:32] <nhm> everything roles upstream? wait, that's not right...
[1:36] <sagewk> which makes me think that if i change my sync loop to create snaps in fake.sh i'll start seeing it
[1:38] <nhm> sagewk: So you see that behavior in whatever gen is doing, but I don't see that behavior in the tests I did here: http://nhm.ceph.com/movies/mailinglist-tests/btrf-osd0-oneiric-3.4.mpg
[1:39] <sagewk> nhm: yeah...
[1:39] <sagewk> well, maybe.. there are still a lot of seeks, but it's not clear from the movie where they are
[1:40] <sagewk> it might be the same problem, but with different placement
[1:41] <nhm> sagewk: just eyeballing it, it looks like the increase in seeks in my movie is correlated with the writes down near the beginning of the disk.
[1:41] <nhm> when the seeks pick up, it looks like the frequency of those writes do too.
[1:42] <sagewk> yeah
[1:48] <nhm> sagewk: have you looked at the blkparse output for your run?
[1:49] <nhm> sagewk: we should be able to see what kind of writes those seeks are.
[1:53] <nhm> I'm seeing a bunch of stuff like this in the blkparse output:
[1:53] <nhm> 8,16 1 251 0.628116476 3212 Q WS 12776832 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 252 0.628120117 3212 G WS 12776832 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 253 0.628122140 3212 Q WS 14873984 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 254 0.628125801 3212 G WS 14873984 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 255 0.628127857 3212 Q WS 12776960 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 256 0.628129321 3212 M WS 12776960 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 257 0.628130470 3212 Q WS 14874112 + 128 [btrfs-submit-1]
[1:53] <nhm> 8,16 1 258 0.628131597 3212 M WS 14874112 + 128 [btrfs-submit-1]
[2:00] * joao (~JL@aon.hq.newdream.net) Quit (Remote host closed the connection)
[2:03] <sagewk> haven't looked, nope.
[2:04] <nhm> sagewk: gen.sdb.blktrace.human on burnupi62
[2:05] * joshd (~joshd@aon.hq.newdream.net) Quit (Quit: Leaving.)
[2:06] <nhm> btw, the sprint official ends at midnight right? :P
[2:06] <yanzheng> spurious writes might be from the extent tree. modifying extent tree is recursion.
[2:08] * Tv_ (~tv@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[2:13] * ecawthon (~eleanor@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[2:19] * izdubar (~MT@c-71-198-138-155.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[2:21] <nhm> sage: going back and looking at some of my previous results, I see something that looks a lot more like your gen results, but it's doing 256k writes: http://nhm.ceph.com/movies/wip-throttle/256k-flusher-2threads-btrfs-osd0.mpg
[2:22] * lofejndif (~lsqavnbok@83TAAGF36.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)
[2:53] * aa (~aa@r200-40-114-26.ae-static.anteldata.net.uy) Quit (Remote host closed the connection)
[3:01] * BManojlovic (~steki@212.200.243.232) Quit (Ping timeout: 480 seconds)
[3:09] * yanzheng (~zhyan@101.84.4.69) Quit (Ping timeout: 480 seconds)
[3:17] * adjohn (~adjohn@50-0-133-101.dsl.static.sonic.net) Quit (Quit: adjohn)
[3:28] * aa (~aa@r190-135-71-47.dialup.adsl.anteldata.net.uy) has joined #ceph
[3:44] * s[X]_ (~sX]@ppp59-167-154-113.static.internode.on.net) Quit (Remote host closed the connection)
[3:49] * aa (~aa@r190-135-71-47.dialup.adsl.anteldata.net.uy) Quit (Ping timeout: 480 seconds)
[3:56] * adjohn (~adjohn@50-0-133-101.dsl.static.sonic.net) has joined #ceph
[3:56] * adjohn (~adjohn@50-0-133-101.dsl.static.sonic.net) Quit ()
[4:08] * aa (~aa@r190-135-71-47.dialup.adsl.anteldata.net.uy) has joined #ceph
[4:25] <elder> sagewk, your patch fixing the reference counting is broken. I will have to dig into it a bit more, but I was afraid of this. The code was wrong, but the environment in which it ran made it work OK (most of the time?). Fixing the code likely needs a corresponding fix elsewhere.
[4:26] <elder> I get a null pointer dereference almost immediately when running the rbd task (but the ceph task seems to have finished).
[4:27] <elder> Kind of a bummer.
[4:29] * renzhi (~renzhi@180.169.73.90) has joined #ceph
[4:34] * chutzpah (~chutz@216.174.109.254) Quit (Quit: Leaving)
[4:36] * aa (~aa@r190-135-71-47.dialup.adsl.anteldata.net.uy) Quit (Ping timeout: 480 seconds)
[4:46] * renzhi (~renzhi@180.169.73.90) Quit (Ping timeout: 480 seconds)
[4:46] * renzhi (~renzhi@69.163.36.54) has joined #ceph
[5:03] * yanzheng (~zhyan@114.87.243.120) has joined #ceph
[5:13] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[5:19] * renzhi (~renzhi@69.163.36.54) Quit (Ping timeout: 480 seconds)
[5:32] * renzhi (~renzhi@180.169.73.90) has joined #ceph
[6:18] * bchrisman (~Adium@c-76-103-130-94.hsd1.ca.comcast.net) has joined #ceph
[6:31] * renzhi (~renzhi@180.169.73.90) Quit (Ping timeout: 480 seconds)
[6:45] * s[X]_ (~sX]@ppp59-167-157-96.static.internode.on.net) has joined #ceph
[6:48] * dmick (~dmick@aon.hq.newdream.net) has left #ceph
[7:19] * MarkDude (~MT@c-71-198-138-155.hsd1.ca.comcast.net) has joined #ceph
[7:27] * bchrisman1 (~Adium@c-76-103-130-94.hsd1.ca.comcast.net) has joined #ceph
[7:33] * bchrisman (~Adium@c-76-103-130-94.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[8:02] * aa (~aa@r190-135-71-47.dialup.adsl.anteldata.net.uy) has joined #ceph
[8:10] * yanzheng (~zhyan@114.87.243.120) Quit (Ping timeout: 480 seconds)
[8:26] * yanzheng (~zhyan@101.84.133.102) has joined #ceph
[8:45] * gregaf1 (~Adium@aon.hq.newdream.net) has joined #ceph
[8:49] * mkampe (~markk@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[8:49] * gregaf (~Adium@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[8:52] * sagewk (~sage@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[8:53] * gregaf1 (~Adium@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[8:54] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[8:54] * sjust (~sam@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[8:55] * yehudasa (~yehudasa@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[8:55] * yehudasa (~yehudasa@aon.hq.newdream.net) has joined #ceph
[8:57] * sjust (~sam@aon.hq.newdream.net) has joined #ceph
[8:57] * gregaf (~Adium@aon.hq.newdream.net) has joined #ceph
[8:58] * mkampe (~markk@aon.hq.newdream.net) has joined #ceph
[9:03] * sagewk (~sage@aon.hq.newdream.net) has joined #ceph
[9:03] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[9:05] * gregaf (~Adium@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[9:06] * yehudasa (~yehudasa@aon.hq.newdream.net) Quit (Ping timeout: 480 seconds)
[9:06] * gregaf (~Adium@aon.hq.newdream.net) has joined #ceph
[9:06] * yehudasa (~yehudasa@aon.hq.newdream.net) has joined #ceph
[9:53] * aa (~aa@r190-135-71-47.dialup.adsl.anteldata.net.uy) Quit (Ping timeout: 480 seconds)
[10:03] * s[X]_ (~sX]@ppp59-167-157-96.static.internode.on.net) Quit (Remote host closed the connection)
[10:03] * MarkDude (~MT@c-71-198-138-155.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[10:28] * s[X]_ (~sX]@ppp59-167-157-96.static.internode.on.net) has joined #ceph
[10:37] * s[X]_ (~sX]@ppp59-167-157-96.static.internode.on.net) Quit (Remote host closed the connection)
[11:09] * dwm_ (~dwm@2001:ba8:0:1c0:225:90ff:fe08:9150) Quit (Ping timeout: 480 seconds)
[11:26] * yanzheng (~zhyan@101.84.133.102) Quit (Ping timeout: 480 seconds)
[11:32] * aliguori (~anthony@222.128.202.2) has joined #ceph
[11:40] * BManojlovic (~steki@212.200.243.232) has joined #ceph
[11:53] * aliguori (~anthony@222.128.202.2) Quit (Remote host closed the connection)
[13:04] * Ryan_Lane (~Adium@p54834911.dip.t-dialin.net) has joined #ceph
[13:26] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[14:09] * yanzheng (~zhyan@101.82.138.3) has joined #ceph
[14:16] * s[X]_ (~sX]@ppp59-167-157-96.static.internode.on.net) has joined #ceph
[14:39] * BManojlovic (~steki@212.200.243.232) Quit (Ping timeout: 480 seconds)
[14:42] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[14:44] * Ryan_Lane1 (~Adium@p54834911.dip.t-dialin.net) has joined #ceph
[14:44] * Ryan_Lane (~Adium@p54834911.dip.t-dialin.net) Quit (Read error: Connection reset by peer)
[14:48] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Quit: later)
[14:49] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[14:59] * Ryan_Lane (~Adium@p548345B5.dip.t-dialin.net) has joined #ceph
[15:04] * Ryan_Lane1 (~Adium@p54834911.dip.t-dialin.net) Quit (Ping timeout: 480 seconds)
[15:17] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Quit: later)
[15:25] * Ryan_Lane1 (~Adium@p3E9D2F65.dip.t-dialin.net) has joined #ceph
[15:30] * Ryan_Lane (~Adium@p548345B5.dip.t-dialin.net) Quit (Ping timeout: 480 seconds)
[16:45] * yanzheng (~zhyan@101.82.138.3) Quit (Ping timeout: 480 seconds)
[17:17] * lofejndif (~lsqavnbok@1RDAACCG2.tor-irc.dnsbl.oftc.net) has joined #ceph
[18:33] * s[X]_ (~sX]@ppp59-167-157-96.static.internode.on.net) Quit (Remote host closed the connection)
[18:34] * hijacker (~hijacker@213.91.163.5) Quit (Ping timeout: 480 seconds)
[18:43] * hijacker (~hijacker@213.91.163.5) has joined #ceph
[18:44] * Ryan_Lane1 (~Adium@p3E9D2F65.dip.t-dialin.net) Quit (Quit: Leaving.)
[21:51] * Ryan_Lane (~Adium@p3E9D2F65.dip.t-dialin.net) has joined #ceph
[21:52] * BManojlovic (~steki@212.200.243.232) has joined #ceph
[22:01] * aa (~aa@r186-52-183-6.dialup.adsl.anteldata.net.uy) has joined #ceph
[22:29] * mtk (~mtk@ool-44c35967.dyn.optonline.net) Quit (Remote host closed the connection)
[23:04] * lofejndif (~lsqavnbok@1RDAACCG2.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)
[23:05] * lofejndif (~lsqavnbok@9KCAAFVJY.tor-irc.dnsbl.oftc.net) has joined #ceph
[23:08] * Ryan_Lane (~Adium@p3E9D2F65.dip.t-dialin.net) Quit (Quit: Leaving.)
[23:08] * aa (~aa@r186-52-183-6.dialup.adsl.anteldata.net.uy) Quit (Ping timeout: 480 seconds)
[23:12] * lofejndif (~lsqavnbok@9KCAAFVJY.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.