#ceph IRC Log

Index

IRC Log for 2013-02-03

Timestamps are in GMT/BST.

[0:10] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) has left #ceph
[0:19] * BillK (~BillK@124-169-94-186.dyn.iinet.net.au) has joined #ceph
[0:20] * Akendo (~akendo@cable-89-16-142-69.cust.telecolumbus.net) has joined #ceph
[0:58] * Akendo (~akendo@cable-89-16-142-69.cust.telecolumbus.net) Quit (Ping timeout: 480 seconds)
[1:07] <nz_monkey> nhm: I am looking forward to your next blog post :)
[1:30] * Matt (matt@matt.netop.oftc.net) has joined #ceph
[2:03] * Kioob (~kioob@luuna.daevel.fr) Quit (Quit: Leaving.)
[2:15] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[2:16] * leseb (~leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) has joined #ceph
[2:34] * LeaChim (~LeaChim@027ee384.bb.sky.com) Quit (Ping timeout: 480 seconds)
[3:59] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[4:02] * wschulze1 (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) Quit (Quit: Leaving.)
[4:05] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) has joined #ceph
[4:28] * gucki (~smuxi@84-72-10-109.dclient.hispeed.ch) Quit (Remote host closed the connection)
[4:44] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[4:45] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) Quit (Remote host closed the connection)
[4:46] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[4:57] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) Quit (Quit: Leaving.)
[5:35] * gaveen (~gaveen@112.135.4.210) has joined #ceph
[5:51] <via> are there performance advantages to journal aio, or any good reasons not to use it?
[6:10] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) Quit (Quit: Leaving.)
[7:17] * joyhu (~joyhu@59.51.235.14) has joined #ceph
[7:17] * joyhu (~joyhu@59.51.235.14) has left #ceph
[7:18] * joyhu (~joyhu@59.51.235.14) has joined #ceph
[7:18] * joyhu (~joyhu@59.51.235.14) has left #ceph
[7:42] * elder (~elder@c-71-195-31-37.hsd1.mn.comcast.net) Quit (Ping timeout: 480 seconds)
[8:01] * ido (~ido@00014f21.user.oftc.net) has joined #ceph
[9:28] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[9:28] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[9:38] * Matt (matt@matt.netop.oftc.net) Quit (Ping timeout: 480 seconds)
[9:40] * loicd (~loic@stat-217-145-38-50.xdsl.toledo.be) Quit (Quit: Leaving.)
[9:47] * sjustlaptop (~sam@62.206.227.172) has joined #ceph
[10:01] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[10:32] * Ryan_Lane (~Adium@2001:67c:11f0:cafe:2515:5c59:b5d9:922f) has joined #ceph
[10:40] * Ryan_Lane (~Adium@2001:67c:11f0:cafe:2515:5c59:b5d9:922f) Quit (Quit: Leaving.)
[10:41] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) has joined #ceph
[10:41] * ChanServ sets mode +o scuttlemonkey
[10:56] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[10:58] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has left #ceph
[11:07] * TMM (~hp@535240C7.cm-6-3b.dynamic.ziggo.nl) Quit (Remote host closed the connection)
[11:07] * madkiss (~madkiss@178.188.60.118) has joined #ceph
[11:08] * LeaChim (~LeaChim@027ee384.bb.sky.com) has joined #ceph
[11:09] * sleinen (~Adium@2001:620:0:25:ec9d:19fc:468e:1da8) has joined #ceph
[11:14] * gucki (~smuxi@84-72-10-109.dclient.hispeed.ch) has joined #ceph
[11:14] <gucki> hi there
[11:16] * lightspeed (~lightspee@fw-carp-wan.ext.lspeed.org) Quit (Ping timeout: 480 seconds)
[11:16] * loicd (~loic@2001:67c:11f0:cafe:120b:a9ff:feb7:cce0) has joined #ceph
[11:22] * madkiss (~madkiss@178.188.60.118) Quit (Quit: Leaving.)
[11:35] * sjustlaptop (~sam@62.206.227.172) Quit (Quit: Leaving.)
[11:36] * sjustlaptop (~sam@62.206.227.172) has joined #ceph
[11:37] * sjustlaptop (~sam@62.206.227.172) Quit (Remote host closed the connection)
[11:46] * TMM (~hp@535240C7.cm-6-3b.dynamic.ziggo.nl) has joined #ceph
[12:04] * joyhu (~joyhu@59.51.235.14) has joined #ceph
[12:04] * joyhu (~joyhu@59.51.235.14) has left #ceph
[12:18] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) Quit (Quit: This computer has gone to sleep)
[12:36] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) has joined #ceph
[12:36] * ChanServ sets mode +o scuttlemonkey
[13:01] * loicd (~loic@2001:67c:11f0:cafe:120b:a9ff:feb7:cce0) Quit (Quit: Leaving.)
[13:02] * Akendo (~akendo@cable-89-16-142-69.cust.telecolumbus.net) has joined #ceph
[13:11] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) Quit (Quit: This computer has gone to sleep)
[13:23] * Akendo (~akendo@cable-89-16-142-69.cust.telecolumbus.net) Quit (Ping timeout: 480 seconds)
[13:26] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) has joined #ceph
[13:26] * ChanServ sets mode +o scuttlemonkey
[13:37] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[13:38] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has left #ceph
[14:02] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) Quit (Quit: This computer has gone to sleep)
[14:02] * loicd (~loic@2001:67c:11f0:cafe:120b:a9ff:feb7:cce0) has joined #ceph
[14:12] * loicd1 (~loic@214.69.ipv4.fosdem.net) has joined #ceph
[14:12] * SkyEye (~gaveen@112.135.34.88) has joined #ceph
[14:17] * Ryan_Lane (~Adium@2001:67c:11f0:cafe:fcaf:37c8:206c:ecd9) has joined #ceph
[14:17] * loicd (~loic@2001:67c:11f0:cafe:120b:a9ff:feb7:cce0) Quit (Read error: No route to host)
[14:19] * gaveen (~gaveen@112.135.4.210) Quit (Ping timeout: 480 seconds)
[14:22] * loicd1 (~loic@214.69.ipv4.fosdem.net) Quit (Quit: Leaving.)
[14:23] * Ryan_Lane (~Adium@2001:67c:11f0:cafe:fcaf:37c8:206c:ecd9) Quit (Quit: Leaving.)
[14:24] * lightspeed (~lightspee@fw-carp-wan.ext.lspeed.org) has joined #ceph
[14:26] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[14:31] * loicd (~loic@2001:67c:11f0:cafe:120b:a9ff:feb7:cce0) has joined #ceph
[14:31] * loicd (~loic@2001:67c:11f0:cafe:120b:a9ff:feb7:cce0) Quit ()
[14:32] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) has joined #ceph
[14:39] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) Quit (Quit: Leaving.)
[14:50] * scalability-junk (~stp@188-193-201-35-dynip.superkabel.de) has joined #ceph
[14:50] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) has joined #ceph
[15:03] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) Quit (Quit: Leaving.)
[15:04] * leseb (~leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Remote host closed the connection)
[15:41] * maxia (~sensa@114.91.100.100) has joined #ceph
[15:41] * leseb (~leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) has joined #ceph
[15:48] * BillK (~BillK@124-169-94-186.dyn.iinet.net.au) Quit (Quit: Leaving)
[15:53] * leseb (~leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Remote host closed the connection)
[15:55] * madkiss (~madkiss@089144192242.atnat0001.highway.a1.net) has joined #ceph
[15:59] * madkiss1 (~madkiss@194.112.182.213) has joined #ceph
[16:03] * madkiss (~madkiss@089144192242.atnat0001.highway.a1.net) Quit (Ping timeout: 480 seconds)
[16:06] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[16:12] * madkiss1 (~madkiss@194.112.182.213) Quit (Quit: Leaving.)
[16:15] * SkyEye (~gaveen@112.135.34.88) Quit (Quit: Leaving)
[16:18] * Akendo (~akendo@cable-89-16-142-69.cust.telecolumbus.net) has joined #ceph
[16:34] * janos (~janos@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[16:53] * Akendo (~akendo@cable-89-16-142-69.cust.telecolumbus.net) has left #ceph
[17:09] * ScOut3R (~ScOut3R@gprs5e1bac29.pool.t-umts.hu) has joined #ceph
[17:12] * Ryan_Lane (~Adium@10.113-78-194.adsl-static.isp.belgacom.be) has joined #ceph
[17:13] * ScOut3R (~ScOut3R@gprs5e1bac29.pool.t-umts.hu) Quit (Remote host closed the connection)
[17:19] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) has joined #ceph
[17:19] * ChanServ sets mode +o scuttlemonkey
[17:28] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) has joined #ceph
[17:31] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[17:34] <absynth_> sjust: so how's the place?
[17:44] <maxia> we have a cluster of 5 ceph nodes. we need to reboot one of the nodes
[17:44] <maxia> the node has 10 osds
[17:45] <maxia> how do we prevent the cluster from going into "doom" mode during this process?
[17:45] <maxia> it takes about 7 minutes for the server to reboot
[17:47] <absynth_> ceph osd noout
[17:48] <maxia> thanks
[17:48] <absynth_> prevents backfill
[17:48] <absynth_> but!
[17:48] <maxia> ahh
[17:48] <absynth_> if you have the default pre 0.56 crushmap, you might have PGs that are exclusively on that host
[17:48] <absynth_> meaning: you might encounter unfound PGs
[17:48] <nhm> absynth_: ah, is Sam over there now?
[17:49] <absynth_> only since .56 does ceph distribute replicas across nodes, not OSDs
[17:49] <maxia> root@s6:~# ceph -v
[17:49] <maxia> ceph version 0.55.1-1-g7c7469a (7c7469a19b0d563a448486adce9f326c6e5bd66d)
[17:49] <absynth_> if i were you, i'd dry-run this before the actual reboot
[17:49] <absynth_> nhm: yeah, landed this morning - we'll meet him tomorrow morning
[17:49] <maxia> we're canceling the reboot
[17:49] <maxia> starting new ceph cluster, #5
[17:49] <maxia> joy.
[17:49] * stxShadow (~Jens@ip-178-201-147-146.unitymediagroup.de) has joined #ceph
[17:50] <absynth_> well, a dry run would probably take only about 2 mins
[17:51] <maxia> in a perfect worlds
[17:51] <absynth_> ceph osd noout; service ceph stop; see what happens
[17:51] <maxia> we stopped doing that
[17:51] <absynth_> if everything breaks, just restart
[17:52] <maxia> what usually happens is a full shutdown for 48 hours
[17:52] <maxia> causing a near shutdown of our entire company
[17:52] <absynth_> i smell a certain frustration
[17:52] <maxia> ok.
[17:52] <absynth_> what are you using ceph for?
[17:52] <maxia> storing a lot of files. Yes it's frustration. Do you really think running it dryrun is safe?
[17:52] <maxia> I'll do it if someone here will back me up on it
[17:53] <absynth_> from my experience, the following two things can happen
[17:53] <maxia> I have a readonly cluster, #3, that we can use
[17:53] <absynth_> a) you will encounter unfound PGs -> that data is inaccessible for the length of the dryrun
[17:53] <absynth_> b) slow requests - as soon as you see these, restart ceph again ASAP
[17:53] <absynth_> c) further OSD crashes due to bugs -> restart OSDs and let the cluster regain stability
[17:54] <maxia> it's c) that kills us
[17:54] <absynth_> yeah, i know, we have the same issue
[17:54] <maxia> weeks
[17:54] <absynth_> do you want to know a hotfix that works for us?
[17:54] <maxia> Yeah
[17:54] <absynth_> while true; do /etc/init.d/ceph osd start; done
[17:54] <absynth_> in a screen
[17:55] <maxia> Do you work for Inktank? Is that an Inktank suggestion?
[17:55] <absynth_> that doesn't help for persistent asserts but for "incidental" crashes it increases stability
[17:55] <absynth_> no. that is a community member suggestion
[17:55] <maxia> that community member, and probably you as well, may have gotten lucky with that
[17:55] <absynth_> that community member _is_ me
[17:56] <absynth_> it works for the kind of crash that usually escalates quickly
[17:56] <absynth_> it doesn't help for crashes due to bugs (i.e. inconsistent pgs, uncovered inconsistencies etc.)
[17:56] <nhm> absynth_: Sam is really good. He should be able to take good care of you guys.
[17:57] <absynth_> nhm: yeah, i'm a lot less worried than i used to be
[17:57] <nhm> absynth_: well, you guys have been through a lot. :)
[17:57] <absynth_> maxia: either way, we are in the same situation as you. ceph outage == massive shitstorm
[17:57] <absynth_> so i really, really feel your pain
[17:58] <maxia> 48 hours, 3 times now :(
[17:58] <absynth_> i can only share the limited ideas we have to mitigate the usual issues
[17:58] <absynth_> what kernels are you running?
[17:58] <maxia> 3.2.0-3-amd64 and up
[18:00] <maxia> absynth_, what kind of servers are you running, btw?
[18:00] <nhm> maxia: unfortunately recovery isn't really my area. :/
[18:00] <absynth_> maxia: what do you mean, hardware-like?
[18:00] <maxia> absynth_, yeah. are you grabbing them from a vendor like dell?
[18:01] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[18:01] <absynth_> no, we use www.thomas-krenn.com
[18:01] <absynth_> german vendor, been working with them for 10y+
[18:01] <absynth_> essential supermicro barebones
[18:02] <absynth_> +ly
[18:02] <maxia> make sure the disk cache is on :)
[18:02] <nhm> absynth_: how big of a pressence does bull have over there?
[18:02] <absynth_> bull? no noticable commercial presence
[18:03] <absynth_> maxia: we have SSD caches (LSI + CacheCade software)
[18:03] <absynth_> nhm: i once worked with a bull cluster at university, that was 60 supermicro barebones. outside that area, i have never seen them
[18:04] <nhm> absynth_: I think they are mostly HPC, but afaik they are trying to expand into cloud.
[18:04] <absynth_> nhm: yeah, their HPC presence was more noticeable, but the cluster i was working with sucked so hard it could suck a golf ball through a garden hose
[18:04] <absynth_> crap stuff
[18:05] <nhm> absynth_: I've never worked with any of their stuff. Just curious since they (seem) to have a big lustre pressence over in Europe.
[18:09] <absynth_> i think i'm below the radar to reliably answer the question
[18:09] <absynth_> as a hardware vendor for individual servers, they are not present here
[18:10] <absynth_> maxia: anyway. i'd try what happens without actually rebooting the machine
[18:10] <nhm> absynth_: good to know. I've been kind of wondering who the popular system integrators are over there.
[18:10] <absynth_> i had a reboot where the machine did an fsck after reboot and we were sitting there sweating blood and tears
[18:10] <nhm> absynth_: IE who might be able to actually put together good/reliable ceph servers.
[18:12] <absynth_> queried you
[18:13] <absynth_> anyway, i tend to think that Bull is a large-cluster company (with large being anything between 50 and 500 nodes)
[18:13] <absynth_> i've seen rackable/SGI clusters in production here, too
[18:13] <absynth_> but that's a completely different cup of tea
[18:13] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[18:18] * drokita1 (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) has joined #ceph
[18:20] * drokita (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) Quit (Ping timeout: 480 seconds)
[18:22] * drokita (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) has joined #ceph
[18:22] <nhm> absynth_: yeah, we've got a big SGI office near here.
[18:22] <nhm> absynth_: Cray too.
[18:26] * drokita1 (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) Quit (Ping timeout: 480 seconds)
[18:26] * loicd (~loic@90.84.144.218) has joined #ceph
[18:40] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) Quit (Quit: This computer has gone to sleep)
[18:42] * loicd (~loic@90.84.144.218) Quit (Ping timeout: 480 seconds)
[18:43] * The_Bishop (~bishop@2001:470:50b6:0:6157:b33e:c40d:437b) has joined #ceph
[18:47] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[18:47] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has left #ceph
[18:52] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) has joined #ceph
[19:10] * stxShadow (~Jens@ip-178-201-147-146.unitymediagroup.de) has left #ceph
[19:14] * sleinen (~Adium@2001:620:0:25:ec9d:19fc:468e:1da8) Quit (Quit: Leaving.)
[19:14] * sleinen (~Adium@130.59.94.157) has joined #ceph
[19:15] * loicd (~loic@2a01:e35:2eba:db10:accc:dd91:8c6d:d542) has joined #ceph
[19:22] * sleinen (~Adium@130.59.94.157) Quit (Ping timeout: 480 seconds)
[19:26] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[19:27] * Cube (~Cube@cpe-76-95-223-199.socal.res.rr.com) Quit ()
[19:29] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) has joined #ceph
[19:31] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) Quit ()
[20:02] * wschulze (~wschulze@cpe-98-14-23-162.nyc.res.rr.com) Quit (Quit: Leaving.)
[20:25] * GBurton (~GBurton@46-65-88-85.zone16.bethere.co.uk) has joined #ceph
[20:31] * KevinPerks (~Adium@cpe-066-026-239-136.triad.res.rr.com) Quit (Quit: Leaving.)
[20:34] * sleinen (~Adium@user-28-14.vpn.switch.ch) has joined #ceph
[20:42] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has joined #ceph
[20:51] * andreask (~andreas@h081217068225.dyn.cm.kabsi.at) has left #ceph
[20:55] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) has joined #ceph
[20:55] * ChanServ sets mode +o scuttlemonkey
[21:02] * sleinen (~Adium@user-28-14.vpn.switch.ch) Quit (Quit: Leaving.)
[21:09] * leseb (~leseb@5ED17881.cm-7-2b.dynamic.ziggo.nl) has joined #ceph
[21:20] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) has joined #ceph
[21:29] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) has joined #ceph
[21:30] * sleinen1 (~Adium@2001:620:0:25:4957:e7a:8624:40b9) has joined #ceph
[21:33] * sleinen2 (~Adium@217-162-132-182.dynamic.hispeed.ch) has joined #ceph
[21:34] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) Quit (Quit: Leaving.)
[21:34] * sleinen3 (~Adium@2001:620:0:26:c14d:5167:3e0c:e5f0) has joined #ceph
[21:37] * sleinen (~Adium@217-162-132-182.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[21:40] * sleinen1 (~Adium@2001:620:0:25:4957:e7a:8624:40b9) Quit (Ping timeout: 480 seconds)
[21:41] * sleinen2 (~Adium@217-162-132-182.dynamic.hispeed.ch) Quit (Ping timeout: 480 seconds)
[22:04] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) Quit (Quit: This computer has gone to sleep)
[22:09] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) has joined #ceph
[22:11] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) Quit ()
[22:18] * ScOut3R (~scout3r@540079A1.dsl.pool.telekom.hu) has joined #ceph
[22:29] * drokita1 (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) has joined #ceph
[22:35] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) has joined #ceph
[22:35] * sleinen3 (~Adium@2001:620:0:26:c14d:5167:3e0c:e5f0) Quit (Quit: Leaving.)
[22:35] * drokita (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) Quit (Ping timeout: 480 seconds)
[22:38] * danieagle (~Daniel@177.133.174.193) has joined #ceph
[22:42] * joshd1 (~jdurgin@2602:306:c5db:310:7dd2:ff03:e100:4283) has joined #ceph
[22:50] * drokita (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) has joined #ceph
[22:56] * drokita1 (~drokita@24-107-180-86.dhcp.stls.mo.charter.com) Quit (Ping timeout: 480 seconds)
[22:58] * Matt (matt@matt.netop.oftc.net) has joined #ceph
[23:01] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[23:03] * nwl (~levine@atticus.yoyo.org) Quit (Quit: leaving)
[23:06] * nwl (~levine@atticus.yoyo.org) has joined #ceph
[23:06] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[23:07] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[23:08] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[23:11] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) has joined #ceph
[23:12] * scuttlemonkey (~scuttlemo@217.64.252.30.mactelecom.net) has joined #ceph
[23:12] * ChanServ sets mode +o scuttlemonkey
[23:19] * nwl (~levine@atticus.yoyo.org) has left #ceph
[23:20] * Matt (matt@matt.netop.oftc.net) Quit (Ping timeout: 480 seconds)
[23:20] * nwl (~levine@atticus.yoyo.org) has joined #ceph
[23:22] * lx0 is now known as lxo
[23:28] * elder (~elder@c-71-195-31-37.hsd1.mn.comcast.net) has joined #ceph
[23:28] * ChanServ sets mode +o elder
[23:31] * Pagefaulted (~AndChat73@c-67-168-132-228.hsd1.wa.comcast.net) has joined #ceph
[23:34] * madkiss (~madkiss@chello062178057005.20.11.vie.surfer.at) Quit (Quit: Leaving.)
[23:36] * BillK (~BillK@124.150.34.96) has joined #ceph
[23:41] * Pagefaulted (~AndChat73@c-67-168-132-228.hsd1.wa.comcast.net) Quit (Quit: Bye)
[23:41] * elder (~elder@c-71-195-31-37.hsd1.mn.comcast.net) Quit (Quit: Leaving)
[23:43] * KindOne (~KindOne@h96.21.131.174.dynamic.ip.windstream.net) Quit (Ping timeout: 480 seconds)
[23:44] * KindOne (KindOne@h166.214.89.75.dynamic.ip.windstream.net) has joined #ceph
[23:46] * danieagle (~Daniel@177.133.174.193) Quit (Quit: Inte+ :-) e Muito Obrigado Por Tudo!!! ^^)
[23:52] * scalability-junk (~stp@188-193-201-35-dynip.superkabel.de) Quit (Quit: Leaving)
[23:57] * leseb (~leseb@5ED17881.cm-7-2b.dynamic.ziggo.nl) Quit (Remote host closed the connection)
[23:59] * Ryan_Lane (~Adium@10.113-78-194.adsl-static.isp.belgacom.be) Quit (Ping timeout: 480 seconds)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.