#ceph IRC Log


IRC Log for 2012-10-02

Timestamps are in GMT/BST.

[0:00] <gregaf> unfortunately I think the next step is heap profiling, which is described at http://ceph.com/wiki/Memory_Profiling
[0:01] <kyle_> ceph version 0.50-362-g1cd89d1 (commit:1cd89d1cdd16957eb1fd3e2975afd3a67b242e2d)
[0:01] <gregaf> actually, the possibility that you have this profiling on is the only idea so far that I can see doing it
[0:01] <gregaf> oh, hmm, you aren't on a tagged version
[0:01] <gregaf> (doubt it matters, but fyi)
[0:03] <tziOm> What is the schedule for the cephfs to become stable?
[0:03] <kyle_> okay. is there any way to know if it is currently on?
[0:04] <gregaf> if you run the "heap dump" thing and it produces useful output, it's on
[0:04] <gregaf> actually, also run the "heap stats" command and provide that output regardless
[0:13] <kyle_> hmm those commands should generate a log file correct?
[0:14] <tziOm> How would cephfs be for storing Maildir?
[0:14] <gregaf> kyle_: you'll see the most useful output if you're watching "ceph -w" in a separate window
[0:14] <gregaf> tziOm: probably not great, especially since CephFS isn't considered production quality yet
[0:15] <gregaf> but I believe I've heard of people doing it
[0:15] <tziOm> Is anyone working on some mail storage in rados?
[0:15] <tziOm> ofcorse someone is.. but are anyone doing it out in the open?
[0:16] <gregaf> that's an idea I've heard about persistently, but I'm not sure anybody's actually doing it
[0:16] <gregaf> I think wido was the one who's discussed it most seriously?
[0:16] <tziOm> About DBmailer and so on?
[0:16] <edv> Hi all im trying to setup an inital cluster with two nodes for testing, and when i run the command: mkcephfs -a -c /etc/ceph/ceph.conf -k ceph.keyring i get the following error: OSD::mkfs: FileStore::mkfs failed with error -2
[0:17] <tziOm> dbmail its called
[0:17] <edv> here is the full output of the command: https://gist.github.com/3814796
[0:17] <edv> What am i doing wrong here?
[0:18] <tziOm> What about file size and ceph? How much space will it use for a 1k file?
[0:19] <gregaf> tziOm: there'll be some overhead for the first 1k file, but asymptotically it will use up as much disk space as the underlying FS requires for that file
[0:19] <gregaf> so probably either 1k or 4k (multiplied by replication, of course)
[0:19] <tziOm> sure
[0:19] <phantomcircuit> im trying to setup a cluster (on one laptop)
[0:19] <phantomcircuit> https://privatepaste.com/download/f5216aca4a
[0:19] <phantomcircuit> running ubuntu 12.04
[0:19] <tziOm> How would you guys think if starting a project to inject mail into object storage?
[0:20] <gregaf> edv: does the /data/osd.1 directory exist on that node?
[0:20] <gregaf> go for it tziOm
[0:20] <edv> gregaf: they do,
[0:20] <edv> root@ceph-01:/data# ls
[0:20] <edv> mds.1 mon.1 osd.1
[0:20] <gregaf> you should check the mailing list archives, there are some discussions about that topic
[0:21] <tziOm> gregaf, I have seen a couple discussions.. none of what I have seen come to any particular conclution..
[0:21] <gregaf> nope, I don't think there's bean a conclusion, just some background to be aware of
[0:22] <kyle_> 2012-10-01 15:22:00.776229 osd.0 [INF] osd.0tcmalloc heap stats:------------------------------------------------
[0:22] <kyle_> 2012-10-01 15:22:00.776235 osd.0 [INF] MALLOC: 17993875736 (17160.3 MB) Bytes in use by application
[0:22] <kyle_> 2012-10-01 15:22:00.776239 osd.0 [INF] MALLOC: + 100560896 ( 95.9 MB) Bytes in page heap freelist
[0:22] <kyle_> 2012-10-01 15:22:00.776242 osd.0 [INF] MALLOC: + 1354168 ( 1.3 MB) Bytes in central cache freelist
[0:22] <kyle_> 2012-10-01 15:22:00.776245 osd.0 [INF] MALLOC: + 57344 ( 0.1 MB) Bytes in transfer cache freelist
[0:22] <kyle_> 2012-10-01 15:22:00.776248 osd.0 [INF] MALLOC: + 11429168 ( 10.9 MB) Bytes in thread cache freelists
[0:22] <kyle_> 2012-10-01 15:22:00.776251 osd.0 [INF] MALLOC: + 238452736 ( 227.4 MB) Bytes in malloc metadata
[0:22] <kyle_> 2012-10-01 15:22:00.776260 osd.0 [INF] MALLOC: ------------
[0:22] <kyle_> 2012-10-01 15:22:00.776263 osd.0 [INF] MALLOC: = 18345730048 (17495.9 MB) Actual memory used (physical + swap)
[0:22] <kyle_> 2012-10-01 15:22:00.776266 osd.0 [INF] MALLOC: + 135168 ( 0.1 MB) Bytes released to OS (aka unmapped)
[0:22] <kyle_> 2012-10-01 15:22:00.776270 osd.0 [INF] MALLOC: ------------
[0:22] <tziOm> ..maby I have not seen them all, do you have some subjects for my slrn?
[0:22] <kyle_> 2012-10-01 15:22:00.776273 osd.0 [INF] MALLOC: = 18345865216 (17496.0 MB) Virtual address space used
[0:22] <kyle_> 2012-10-01 15:22:00.776277 osd.0 [INF] MALLOC:
[0:22] <kyle_> 2012-10-01 15:22:00.776280 osd.0 [INF] MALLOC: 4184450 Spans in use
[0:22] <kyle_> 2012-10-01 15:22:00.776286 osd.0 [INF] MALLOC: 27 Thread heaps in use
[0:22] <kyle_> 2012-10-01 15:22:00.776289 osd.0 [INF] MALLOC: 4096 Tcmalloc page size
[0:22] <kyle_> 2012-10-01 15:22:00.776293 osd.0 [INF] ------------------------------------------------
[0:22] <kyle_> 2012-10-01 15:22:00.776296 osd.0 [INF] Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
[0:22] <gregaf> sjust: what can cause this output besides the store directory not existing? https://gist.github.com/3814796
[0:22] <kyle_> 2012-10-01 15:22:00.776303 osd.0 [INF] Bytes released to the OS take
[0:23] <sjust> it might mean that the directory is not empty,but is also not a valid osd directory
[0:24] <gregaf> kyle_: okay, so that's indicating the OSD really is using up that much memory
[0:24] <gregaf> which is…pretty bizarre
[0:24] <nhm> BUGS
[0:24] <kyle_> here is one error that keeps repeating:
[0:24] <kyle_> 2012-10-01 15:23:54.145699 mds.0 [ERR] loaded dup inode 10000fccf0e [2,head] v1466296 at /products/thumbs/30000/_product_2129982227_5_thumb_medium.jpg, but inode 10000fccf0e.head v8828299 already exists at /products/thumbs/30000/._product_2129982227_5_thumb_medium.jpg.4FwoHk
[0:24] <gregaf> that's on the MDS
[0:25] <gregaf> although it is interesting
[0:25] <kyle_> also am seeing this:
[0:25] <kyle_> 2012-10-01 15:18:45.475861 osd.0 [WRN] slow request 31.632120 seconds old, received at 2012-10-01 15:18:13.843652: osd_op(client.4110.1:18870425 10001472bad.00000000 [write 0~3760] 0.7dbfc083 snapc 1=[]) currently started
[0:25] <gregaf> that's indicating that the OSD is taking a long time to complete some requests
[0:25] <gregaf> given 25% iowait and the use of swap, that's not surprising
[0:25] <gregaf> okay, I've got two things for you
[0:26] <gregaf> 1) upgrade to a tagged release
[0:26] <gregaf> 2) if memory problems persist, enable debug logging and start up the OSD daemon again with those profiler settings to collect heap dumps we can look at
[0:27] <kyle_> okay. will do. i'm going to wait for this sync to finish so may be a couple days yet. thanks for looking into this for me though. I appreciate it.
[0:27] <tziOm> what are the main uses for the object storage today?
[0:28] <gregaf> phantomcircuit: haven't used mkcephfs much, especially on one node, but try running without -a since it's all local
[0:28] <edv> gregaf: The error was resolved that I was seeing, looks like I needed to create the /var/lib/ceph/{osd,mon,mds}/ceph-$id on the nodes properly on all to not have the command fail
[0:28] <gregaf> kyle_: no problem; I hope we can find out what's happening
[0:28] <edv> phantomcircuit: you may want to ensure that you have those directories
[0:29] <gregaf> tziOm: most deployments today are for either RBD or RGW
[0:29] <tziOm> rdb I see, but what are rgw uses? and through what frontends?
[0:30] <gregaf> Rados Gateway, the S3- and Swift-compatible RESTful access proxy
[0:30] <gregaf> DreamHost Objects is using it publicly; not sure if anybody else is
[0:32] * lxo (~aoliva@9YYAAJFEU.tor-irc.dnsbl.oftc.net) Quit (Quit: later)
[0:34] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[0:38] <tziOm> ok,,,
[0:38] * EmilienM (~EmilienM@195-132-228-252.rev.numericable.fr) has joined #ceph
[0:38] * LarsFronius (~LarsFroni@2a02:8108:3c0:79::3) Quit (Quit: LarsFronius)
[0:40] <phantomcircuit> edv, i do
[0:41] * EmilienM (~EmilienM@195-132-228-252.rev.numericable.fr) has left #ceph
[0:42] <phantomcircuit> it seems liek there is a mismatch between the init script and the expected command line flags
[0:42] <phantomcircuit> i get the same error with sudo /etc/init.d/ceph start osd
[0:43] <gregaf> phantomcircuit: if your ceph.conf isn't in a default location then you need to pass it to the init.d/ceph script
[0:44] <phantomcircuit> ohhh
[0:44] <gregaf> I'm not sure why mkcephfs isn't behaving, though
[0:44] <phantomcircuit> it's because i used the same host value for everything
[0:44] <gregaf> what version are you using?
[0:44] <phantomcircuit> seems sort of obvious now but the error message is extremely confusing
[0:48] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[0:48] <Tv_> phantomcircuit: mkcephfs is confusing, that's why i've been working on a replacement...
[0:49] <phantomcircuit> hmm lets try this again with the ceph packages instead of the ubuntu packages
[1:00] <phantomcircuit> yeah using the official packages it works
[1:00] <phantomcircuit> strangeness
[1:02] * dmick (~dmick@ Quit (Quit: Leaving.)
[1:03] <phantomcircuit> well except now im getting segfaults :(
[1:03] * Cube1 (~Adium@ Quit (Quit: Leaving.)
[1:04] <gregaf> phantomcircuit: where at?
[1:04] <gregaf> and if you just upgraded from the default Ubuntu packages to the ceph.com debian packages you've moved along by…something like 6 months worth of work
[1:04] <gregaf> perhaps 8 or 10
[1:05] <phantomcircuit> the monitor is crashing after gceph tries to connect to it a bajillion times
[1:06] <phantomcircuit> https://privatepaste.com/download/c529f4d616
[1:06] * dmick (~dmick@ has joined #ceph
[1:06] <gregaf> oh, yeah, gceph is dead
[1:06] * kyle_ (~kyle@ip03.foxyf.simplybits.net) Quit (Quit: Leaving)
[1:06] <gregaf> don't use it; I don't think it's even packaged so if you still have it, it's the old version
[1:06] <phantomcircuit> i figured since it didn't work
[1:06] <gregaf> the fact that ti can crash the monitor is bad, though
[1:06] <phantomcircuit> but it's attempts to work totally killed the monitor :/
[1:10] <phantomcircuit> http://ns228969.ovh.net/ceph/
[1:11] <phantomcircuit> have directory listings turned off so that's sort of annoying but it's the best i cna do at the moment
[1:11] <gregaf> phantomcircuit: I can see the directory but not look at the file contents :p
[1:11] <phantomcircuit> well you can you just have to figure out the url yourself
[1:11] <phantomcircuit> lol
[1:11] <phantomcircuit> one sec
[1:13] <gregaf> oh, I see, I can read half the files
[1:13] <phantomcircuit> alright
[1:13] <gregaf> wow, that's...
[1:14] <phantomcircuit> fixed the permissions
[1:14] <phantomcircuit> unintentional fuzzing
[1:14] <gregaf> well, the good news is I *think* this bug is fixed in current master
[1:14] * jlogan1 (~Thunderbi@2600:c00:3010:1:e419:2dcf:b362:1800) Quit (Ping timeout: 480 seconds)
[1:14] <gregaf> man, this is weird
[1:15] <gregaf> it looks like maybe you ran "mon tell *"
[1:15] <phantomcircuit> i might have
[1:15] <gregaf> and it then pasted in all your local dentries
[1:15] <phantomcircuit> im honestly not sure i was just messing around
[1:15] <gregaf> one of which could be properly interpreted as a number
[1:15] <gregaf> I think that's what happened
[1:15] <gregaf> if you look in that log for "mon_command(mon tell"
[1:16] <gregaf> you will see the contents of a command that got sent
[1:16] <gregaf> I don't know what we can do to guard against that, honestly
[1:16] <phantomcircuit> lol yeah
[1:16] <phantomcircuit> that's my home directory
[1:17] * The_Bishop (~bishop@p5DC1134C.dip.t-dialin.net) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[1:17] <gregaf> oh, yep, we can do that
[1:17] <gregaf> looks like it's fixed in master but not argonaut
[1:17] <gregaf> it might be too large to be backported given the current set of users
[1:20] * Cube1 (~cube@ has joined #ceph
[1:20] * maelfius (~mdrnstm@ has joined #ceph
[1:21] * MarkN (~nathan@ has joined #ceph
[1:25] * MarkN (~nathan@ has left #ceph
[1:25] * Karcaw (~evan@68-186-68-219.dhcp.knwc.wa.charter.com) Quit (Ping timeout: 480 seconds)
[1:29] * mgalkiewicz (~mgalkiewi@staticline-31-183-23-218.toya.net.pl) has joined #ceph
[1:35] * Cube1 (~cube@ has left #ceph
[1:36] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Quit: Leseb)
[1:37] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) Quit (Quit: dty)
[1:37] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) has joined #ceph
[1:41] * cblack101 (86868949@ircip3.mibbit.com) Quit (Quit: http://www.mibbit.com ajax IRC Client)
[1:42] * adjohn (~adjohn@ Quit (Quit: adjohn)
[1:46] * MikeMcClurg (~mike@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) Quit (Quit: Leaving.)
[1:46] * Cube1 (~Adium@ has joined #ceph
[1:49] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Ping timeout: 480 seconds)
[1:50] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) has joined #ceph
[1:56] * tziOm (~bjornar@ti0099a340-dhcp0358.bb.online.no) Quit (Remote host closed the connection)
[1:59] * andreask (~andreas@chello062178013131.5.11.vie.surfer.at) Quit (Quit: Leaving.)
[2:11] * Tv_ (~tv@2607:f298:a:607:b899:20f7:e1bb:234c) Quit (Quit: Tv_)
[2:13] * edv (~edjv@107-1-75-186-ip-static.hfc.comcastbusiness.net) Quit (Quit: Leaving.)
[2:36] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) Quit (Quit: dty)
[2:38] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) has joined #ceph
[2:47] * joshd (~joshd@2607:f298:a:607:221:70ff:fe33:3fe3) Quit (Quit: Leaving.)
[2:49] * Karcaw (~evan@68-186-68-219.dhcp.knwc.wa.charter.com) has joined #ceph
[2:51] * grant (~grant@202-173-147-27.mach.com.au) has joined #ceph
[2:51] <grant> Hey guys!
[3:03] * grant (~grant@202-173-147-27.mach.com.au) Quit (Read error: Connection reset by peer)
[3:04] * maelfius (~mdrnstm@ Quit (Quit: Leaving.)
[3:04] * yoshi (~yoshi@p37219-ipngn1701marunouchi.tokyo.ocn.ne.jp) has joined #ceph
[3:07] * adjohn (~adjohn@108-225-130-229.lightspeed.sntcca.sbcglobal.net) has joined #ceph
[3:12] * _are_ (~quassel@2a01:238:4325:ca02::42:4242) has joined #ceph
[3:12] * mdxi_ (~mdxi@74-95-29-182-Atlanta.hfc.comcastbusiness.net) Quit (Read error: Connection reset by peer)
[3:17] * adjohn (~adjohn@108-225-130-229.lightspeed.sntcca.sbcglobal.net) Quit (Read error: Connection reset by peer)
[3:17] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[3:17] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[3:29] * Cube1 (~Adium@ Quit (Ping timeout: 480 seconds)
[3:34] * mdxi (~mdxi@74-95-29-182-Atlanta.hfc.comcastbusiness.net) has joined #ceph
[3:49] * Ryan_Lane (~Adium@ Quit (Quit: Leaving.)
[3:56] * Cube1 (~Adium@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[3:59] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) Quit (Quit: dty)
[4:00] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Ping timeout: 480 seconds)
[4:02] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) has joined #ceph
[4:05] * slang (~slang@207-229-177-80.c3-0.drb-ubr1.chi-drb.il.cable.rcn.com) Quit (Quit: slang)
[4:19] * chutzpah (~chutz@ Quit (Quit: Leaving)
[4:27] * slang (~slang@207-229-177-80.c3-0.drb-ubr1.chi-drb.il.cable.rcn.com) has joined #ceph
[4:46] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) Quit (Ping timeout: 480 seconds)
[4:48] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Quit: zzzzzzzzzzzzzzzzzzzz)
[5:11] * Ryan_Lane (~Adium@c-67-160-217-184.hsd1.ca.comcast.net) has joined #ceph
[5:53] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[5:53] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[6:36] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[6:43] * dmick (~dmick@ Quit (Quit: Leaving.)
[6:50] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[7:01] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[7:01] * jjgalvez (~jjgalvez@cpe-76-175-17-226.socal.res.rr.com) Quit (Read error: Connection reset by peer)
[7:01] * jjgalvez (~jjgalvez@cpe-76-175-17-226.socal.res.rr.com) has joined #ceph
[7:08] * edv (~edjv@50-46-210-159.evrt.wa.frontiernet.net) has joined #ceph
[7:14] * Ryan_Lane1 (~Adium@c-67-160-217-184.hsd1.ca.comcast.net) has joined #ceph
[7:15] * Ryan_Lane (~Adium@c-67-160-217-184.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[7:19] * tomaw (tom@tomaw.netop.oftc.net) Quit (Ping timeout: 600 seconds)
[7:54] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[7:54] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[8:05] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[8:19] * loicd (~loic@magenta.dachary.org) has joined #ceph
[8:22] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[8:39] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[8:39] * loicd (~loic@magenta.dachary.org) has joined #ceph
[8:56] * EmilienM (~EmilienM@ has joined #ceph
[9:20] * mgalkiewicz (~mgalkiewi@staticline-31-183-23-218.toya.net.pl) has left #ceph
[9:21] * BManojlovic (~steki@ has joined #ceph
[9:22] * EmilienM (~EmilienM@ Quit (Ping timeout: 480 seconds)
[9:23] * EmilienM (~EmilienM@ has joined #ceph
[9:30] * EmilienM (~EmilienM@ has left #ceph
[9:31] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:50fa:105d:1298:3d2d) has joined #ceph
[9:44] * BManojlovic (~steki@ Quit (Quit: Ja odoh a vi sta 'ocete...)
[9:45] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:50fa:105d:1298:3d2d) Quit (Quit: LarsFronius)
[9:45] * Leseb (~Leseb@ has joined #ceph
[9:52] * edv (~edjv@50-46-210-159.evrt.wa.frontiernet.net) Quit (Quit: Leaving.)
[10:03] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[10:06] * jbd_ (~jbd_@34322hpv162162.ikoula.com) has joined #ceph
[10:16] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:44be:7b66:9e97:7ce7) has joined #ceph
[10:19] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:44be:7b66:9e97:7ce7) Quit ()
[10:25] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[10:25] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[10:25] * tomaw (tom@tomaw.netop.oftc.net) has joined #ceph
[10:33] * verwilst (~verwilst@d5152FEFB.static.telenet.be) Quit (Ping timeout: 480 seconds)
[10:33] * jjgalvez (~jjgalvez@cpe-76-175-17-226.socal.res.rr.com) Quit (Quit: Leaving.)
[10:35] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[10:41] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:a49d:7f4d:824b:1080) has joined #ceph
[10:43] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:a49d:7f4d:824b:1080) Quit ()
[10:43] * verwilst (~verwilst@d5152FEFB.static.telenet.be) has joined #ceph
[10:59] * grant (~grant@60-240-78-43.static.tpgi.com.au) has joined #ceph
[11:00] * loicd (~loic@LPuteaux-156-16-100-112.w80-12.abo.wanadoo.fr) has joined #ceph
[11:01] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) has joined #ceph
[11:01] <grant> Can anyone help me with non-uniform data distribution across OSDs?
[11:02] * deepsa (~deepsa@ has joined #ceph
[11:17] * grant (~grant@60-240-78-43.static.tpgi.com.au) Quit (Ping timeout: 480 seconds)
[11:25] * yoshi (~yoshi@p37219-ipngn1701marunouchi.tokyo.ocn.ne.jp) Quit (Remote host closed the connection)
[11:35] * LarsFronius_ (~LarsFroni@testing78.jimdo-server.com) has joined #ceph
[11:40] <joao> morning #ceph
[11:43] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) Quit (Ping timeout: 480 seconds)
[11:43] * LarsFronius_ is now known as LarsFronius
[11:44] * loicd (~loic@LPuteaux-156-16-100-112.w80-12.abo.wanadoo.fr) Quit (Quit: Leaving.)
[11:48] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) Quit (Read error: Connection reset by peer)
[11:48] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) has joined #ceph
[12:07] * MikeMcClurg (~mike@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) has joined #ceph
[12:24] * grant (~grant@60-240-78-43.static.tpgi.com.au) has joined #ceph
[13:21] * grant (~grant@60-240-78-43.static.tpgi.com.au) Quit (Ping timeout: 480 seconds)
[13:32] * loicd (~loic@magenta.dachary.org) has joined #ceph
[13:59] * MikeMcClurg (~mike@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) Quit (Quit: Leaving.)
[14:05] * nhorman (~nhorman@2001:470:8:a08:7aac:c0ff:fec2:933b) has joined #ceph
[14:23] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) has joined #ceph
[14:29] * tryggvil_ (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[14:29] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) has joined #ceph
[14:37] * tryggvil_ (~tryggvil@rtr1.tolvusky.sip.is) Quit (Ping timeout: 480 seconds)
[14:37] <slang> joao: morning
[14:40] <joao> morning Sam
[14:40] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) Quit (Read error: Connection reset by peer)
[14:41] <slang> joao: pretty quiet in here for you until all the west coasters arrive I guess
[14:42] * dty (~derek@pool-71-178-175-208.washdc.fios.verizon.net) Quit (Quit: dty)
[14:43] * MikeMcClurg (~mike@firewall.ctxuk.citrix.com) has joined #ceph
[14:49] <joao> slang, it depends; there's actually a lot of guys from Europe around :)
[14:50] <joao> they appear to be idling today though :p
[14:50] <joao> (which may be taken as a good sign!)
[14:50] <slang> joao: octoberfest!
[14:51] <joao> oh yeah... totally forgot about that
[14:51] <joao> lucky Germans
[14:52] <slang> heh
[14:52] <joao> we should have held a ceph meeting there
[14:52] <slang> *nods*
[15:03] * EmilienM (~EmilienM@ has joined #ceph
[15:11] <elder> slang, where are you?
[15:11] <slang> elder: chicago
[15:11] <elder> Ahh, that makes sense.
[15:11] <elder> You are too early to be from California.
[15:12] <slang> yeah
[15:12] <slang> elder: you're in MN?
[15:13] <elder> Yes.
[15:14] <elder> But I'm normally online by 7. That's about 5 hours before a lot of the LA folks get started.
[15:16] * dty (~derek@testproxy.umiacs.umd.edu) has joined #ceph
[15:28] * SvenDowideit (~SvenDowid@203-206-171-38.perm.iinet.net.au) Quit (Ping timeout: 480 seconds)
[15:35] <joao> nhm, around?
[15:43] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) has joined #ceph
[15:46] * loicd (~loic@magenta.dachary.org) Quit (Quit: Leaving.)
[15:48] * cblack101 (86868b4a@ircip4.mibbit.com) has joined #ceph
[16:12] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[16:12] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[16:12] * loicd (~loic@ has joined #ceph
[16:14] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) Quit (Remote host closed the connection)
[16:14] * silversurfer (~silversur@124x35x68x250.ap124.ftth.ucom.ne.jp) has joined #ceph
[16:30] * hijacker_ (~hijacker@ Quit (Quit: Leaving)
[16:34] * bchrisman (~Adium@c-76-103-130-94.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[16:34] * LarsFronius_ (~LarsFroni@testing78.jimdo-server.com) has joined #ceph
[16:37] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) Quit (Read error: Connection reset by peer)
[16:37] * LarsFronius_ is now known as LarsFronius
[16:39] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[16:41] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[16:46] <phantomcircuit> hmm
[16:46] <phantomcircuit> so do all the nodes in the cluster need to have ceph.conf?
[16:46] * hijacker (~hijacker@ has joined #ceph
[16:49] <joao> cool
[16:49] <joao> looks like dreamhost has a speedtest server
[16:53] <joao> phantomcircuit, all daemons must have access to a ceph.conf
[16:53] * joao (~JL@ has left #ceph
[16:53] * joao (~JL@ has joined #ceph
[16:53] <joao> oops
[16:53] <joao> ctrl+w on the wrong window
[16:57] * Leseb (~Leseb@ Quit (Read error: Connection reset by peer)
[16:57] * Leseb (~Leseb@ has joined #ceph
[16:58] <phantomcircuit> joao, i do that all the time
[16:58] <phantomcircuit> it's so annoying
[16:58] <phantomcircuit> also i wouldn't trust dreamhosts spedtest server at all :/
[17:00] <joao> phantomcircuit, truth be told, it performs pretty much the same as any other speedtest server in LA
[17:01] <joao> roughly 8mbps download; 5mbps upload
[17:01] <joao> not bad, considering it's on the other side of the world and with a ping of over 120ms ;)
[17:13] * aliguori (~anthony@cpe-70-123-140-180.austin.res.rr.com) Quit (Quit: Ex-Chat)
[17:25] * senner (~Wildcard@68-113-232-90.dhcp.stpt.wi.charter.com) has joined #ceph
[17:35] * verwilst (~verwilst@d5152FEFB.static.telenet.be) Quit (Quit: Ex-Chat)
[17:40] * loicd (~loic@ Quit (Ping timeout: 480 seconds)
[17:41] * loicd (~loic@ has joined #ceph
[17:41] * jlogan (~Thunderbi@ has joined #ceph
[17:49] <slang> coverity appears to be down for maintenance till tonight
[17:55] * adjohn (~adjohn@108-225-130-229.lightspeed.sntcca.sbcglobal.net) has joined #ceph
[18:00] * gregorg (~Greg@ has joined #ceph
[18:00] * gregorg_taf (~Greg@ Quit (Read error: Connection reset by peer)
[18:01] * Tv_ (~tv@2607:f298:a:607:b899:20f7:e1bb:234c) has joined #ceph
[18:02] * Kioob (~kioob@luuna.daevel.fr) has joined #ceph
[18:05] * gregorg_taf (~Greg@ has joined #ceph
[18:05] * gregorg (~Greg@ Quit (Read error: Connection reset by peer)
[18:07] * gregorg (~Greg@ has joined #ceph
[18:07] * gregorg_taf (~Greg@ Quit (Read error: Connection reset by peer)
[18:07] * MikeMcClurg (~mike@firewall.ctxuk.citrix.com) Quit (Quit: Leaving.)
[18:07] * MikeMcClurg (~mike@firewall.ctxuk.citrix.com) has joined #ceph
[18:08] * gregorg (~Greg@ Quit (Read error: Connection reset by peer)
[18:08] * gregorg (~Greg@ has joined #ceph
[18:09] * aliguori (~anthony@ has joined #ceph
[18:09] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[18:12] * gregorg_taf (~Greg@ has joined #ceph
[18:16] * gregorg (~Greg@ Quit (Ping timeout: 480 seconds)
[18:20] * gregorg_taf (~Greg@ Quit (Ping timeout: 480 seconds)
[18:21] * gregorg (~Greg@ has joined #ceph
[18:24] * gregorg_taf (~Greg@ has joined #ceph
[18:24] * gregorg (~Greg@ Quit (Read error: Connection reset by peer)
[18:28] <sagewk> joao: have a sec to review wip-osd-tree?
[18:28] * gregorg (~Greg@ has joined #ceph
[18:28] * gregorg_taf (~Greg@ Quit (Read error: Connection reset by peer)
[18:29] * edv (~edjv@107-1-75-186-ip-static.hfc.comcastbusiness.net) has joined #ceph
[18:29] <joao> sagewk, sure
[18:29] <joao> looking
[18:29] * gregorg (~Greg@ Quit (Read error: Connection reset by peer)
[18:30] * gregorg (~Greg@ has joined #ceph
[18:32] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Quit: zzzzzzzzzzzzzzzzzzzz)
[18:32] * gregorg (~Greg@ Quit (Read error: Connection reset by peer)
[18:38] * Leseb (~Leseb@ Quit (Quit: Leseb)
[18:40] * loicd (~loic@ Quit (Ping timeout: 480 seconds)
[18:43] * loicd (~loic@ has joined #ceph
[18:45] * EmilienM (~EmilienM@ has left #ceph
[18:55] <joao> sagewk, aside from a couple of comments I left on github, everything looks fine
[18:55] <sagewk> joao: thanks!
[18:55] <joao> also, I'm not that familiar enough with the PGMap to assess if that is a typo or if it is a feature
[18:55] <joao> but I will check the logic to make sure
[18:57] <sagewk> joao: can you look at wip-mon too?
[18:57] <sagewk> (just pushed)
[18:58] <joao> sagewk, give me about 10 minutes; gotta walk the dog before the stand-up, or otherwise I won't be able to do anything with him just being annoying
[18:58] <sagewk> heh ok!
[18:58] <joao> brb
[18:59] * joshd (~joshd@2607:f298:a:607:221:70ff:fe33:3fe3) has joined #ceph
[19:02] * Ryan_Lane1 (~Adium@c-67-160-217-184.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[19:06] * loicd (~loic@ Quit (Ping timeout: 480 seconds)
[19:07] * The_Bishop (~bishop@2001:470:50b6:0:6814:9778:4a30:fd6d) has joined #ceph
[19:08] * gregorg (~Greg@ has joined #ceph
[19:11] * dmick (~dmick@2607:f298:a:607:1a03:73ff:fedd:c856) has joined #ceph
[19:12] <joao> back; looking
[19:12] * joao (~JL@ has left #ceph
[19:12] * joao (~JL@ has joined #ceph
[19:12] <joao> yet again, ctrl+w ftw
[19:12] * jbd_ (~jbd_@34322hpv162162.ikoula.com) has left #ceph
[19:15] * tomaw_ (tom@tomaw.netop.oftc.net) has joined #ceph
[19:17] * Kioob (~kioob@luuna.daevel.fr) Quit (Ping timeout: 480 seconds)
[19:18] * LarsFronius (~LarsFroni@testing78.jimdo-server.com) Quit (Ping timeout: 480 seconds)
[19:20] * loicd (~loic@ has joined #ceph
[19:20] * jjgalvez (~jjgalvez@ has joined #ceph
[19:26] * MikeMcClurg (~mike@firewall.ctxuk.citrix.com) Quit (Quit: Leaving.)
[19:33] * jjgalvez1 (~jjgalvez@ has joined #ceph
[19:34] * loicd (~loic@ Quit (Ping timeout: 480 seconds)
[19:36] <sagewk> joao: oh, right. thanks!
[19:36] <joao> yw :)
[19:37] <dmick> ^W?
[19:39] <joao> dmick, it's an annoyingly common shortcut to close windows
[19:39] * jjgalvez (~jjgalvez@ Quit (Ping timeout: 480 seconds)
[19:40] <joao> the annoying part is due to closing windows from the 'active' application instead of the one you wanted to close the window on
[19:43] * senner (~Wildcard@68-113-232-90.dhcp.stpt.wi.charter.com) Quit (Quit: Leaving.)
[19:44] <Tv_> joao: you want the focus-follows-thought feature
[19:44] <Tv_> aka think-to-focus
[19:45] * dmick lifts mouse and shouts "Computer!" at it
[19:45] <gregaf> I think the focus-follows-thought things actually work if you buy the right gear
[19:45] <gregaf> getting them to do the right input on that focus is still kinda hard
[19:45] * mkampe (~markk@2607:f298:a:607:222:19ff:fe31:b5d3) Quit (Quit: Leaving.)
[19:45] <Tv_> dmick: hey at least that's a more realistic scenario than needing a protocol droid to talk to your astromech droid
[19:46] <gregaf> you only need that if you don't have a computer display
[19:46] <gregaf> though god knows why the projector system can't project text
[19:46] <dmick> PS/2 connectors. They'll never stop plaguing us
[19:47] <gregaf> or how an astromech droid with a projector is cheap enough that you're saving real money by not using a full speaker instead of a tone thing
[19:47] <Tv_> i'm just saying, R2D2 could use a Siri instead of a fax
[19:47] * mkampe (~markk@2607:f298:a:607:222:19ff:fe31:b5d3) has joined #ceph
[19:49] <dmick> someone should speak R2D2 to Siri and see what she says
[19:49] <Tv_> http://www.youtube.com/watch?v=18UmoIu8lII
[19:50] * Cube1 (~Adium@cpe-76-95-223-199.socal.res.rr.com) Quit (Quit: Leaving.)
[19:53] * Ryan_Lane (~Adium@ has joined #ceph
[20:02] <dmick> odd. apt-file isn't indexing ceph-fuse
[20:02] <dmick> wonder why
[20:03] <gregaf> it's in an independent package from all the other binaries, fyi
[20:03] <gregaf> is your install fresh?
[20:03] <dmick> my apt-file cache is fresh
[20:04] <gregaf> or it might have been included in ceph-common in the past and isn't now, so you have an old ceph-fuse that isn't actually being managed any more?
[20:04] <dmick> supposed to catalog files in all the repositories
[20:04] <dmick> (i.e. not installed)
[20:04] <dmick> (although I do in fact also have it installed)
[20:04] * tziOm (~bjornar@ti0099a340-dhcp0358.bb.online.no) has joined #ceph
[20:04] <gregaf> (I'm not really familiar with apt-file, but iirc it's just a reverse lookup)
[20:04] <gregaf> okay, apparently I remember wrong
[20:05] * maelfius (~mdrnstm@ has joined #ceph
[20:05] <Tv_> dmick: using canonical's repos or ours?
[20:06] <dmick> both
[20:06] <dmick> I mean, unless configuring apt-file is different than configuring apt-cache/get
[20:06] <dmick> diggin
[20:07] <Tv_> umm did canonical disable fuse from the ceph packages in 12.04?
[20:08] <gregaf> that'd be an odd one to disable, but they might not have any of the posix stuff in their repos?
[20:08] * Cube1 (~Adium@ has joined #ceph
[20:08] <Tv_> oh the source for 0.41 says ceph-fuse is a separate deb, but http://packages.ubuntu.com/ceph-fuse says it isn't in precise
[20:09] <Tv_> gregaf: they explicitly moved cephfs out of main
[20:10] <Tv_> dmick: i think apt-file lets one version of a package overshadow the other one, so that's why you're not seeing results from our repo
[20:10] <Tv_> dmick: and that 12.04 just doesn't have ceph-fuse
[20:10] <gregaf> Tv_: I just meant disabling ceph-fuse individually would be weird; not including all of cephfs would make sense
[20:10] <Tv_> debian/changelog: * d/rules,d/control: drop ceph-fuse entirely per MIR review
[20:11] <gregaf> but our packages have ceph-fuse in its own package, so shouldn't it be showing up from that even if it was in ceph-common during 0.41?
[20:13] <Tv_> Ignoring source without Contents File:
[20:13] <Tv_> http://ceph.com/debian/dists/precise/Contents-amd64.gz
[20:13] <Tv_> we are not providing the data for apt-file
[20:13] <dmick> that sounds like it might have something to do with it :)
[20:21] <dmick> I'll file an issue
[20:21] <dmick> sounds like a job for Mr. Lowell
[20:22] * EmilienM (~EmilienM@195-132-228-252.rev.numericable.fr) has joined #ceph
[20:23] <dmick> http://tracker.newdream.net/issues/3250
[20:25] <gregaf> dmick: I think you mean a job for glowell, right? ;)
[20:26] <gregaf> (unless that's not who I think it is)
[20:28] <dmick> yes, I was just being obsequiously obfuscatory
[20:28] <glowell> I'll check it out.
[20:28] <dmick> (I assigned you the bug glowell)
[20:29] <gregaf> but if you're obsequiously obfuscatory then they don't get pinged!
[20:29] <dmick> exactly, so we see if he's paying attention! :)
[20:29] <dmick> (no, seriously, better idea. ty.)
[20:42] <dmick> oh speaking of
[20:42] <dmick> does anyone have current status for running ceph/vstart.sh on a 32-bit Ubuntu?
[20:42] <dmick> I was failing last night on my older laptop, and didn't have the energy to diagnose why, but thought I could reproduce on a VM if we're unaware of problems
[20:43] <dmick> someone threw an assertion early in startup; not sure which daemon even, I think probably mon
[20:45] * bchrisman (~Adium@c-76-103-130-94.hsd1.ca.comcast.net) has joined #ceph
[20:48] * andreask (~andreas@chello062178013131.5.11.vie.surfer.at) has joined #ceph
[20:55] <slang> dmick: not sure about the 32-bit part
[20:55] <slang> dmick: I always have to do mkdir -p dev/mon.a dev/mon.b dev/mon.c
[20:56] <slang> dmick: before running it when starting from scratch
[20:56] <dmick> hm. I do not.
[20:56] <dmick> mkdir dev seems to be required
[20:56] <dmick> (which we should probably fix)
[20:56] * pentabular (~sean@adsl-70-231-128-149.dsl.snfc21.sbcglobal.net) has joined #ceph
[20:57] <slang> ah yes looks like it creates the mon.* dirs if dev is there
[20:57] <slang> but complains that dev/mon.a is not there (if dev is not there)
[21:01] * loicd (~loic@magenta.dachary.org) has joined #ceph
[21:17] * LarsFronius (~LarsFroni@tmo-111-123.customers.d1-online.com) has joined #ceph
[21:20] * lofejndif (~lsqavnbok@83TAABCZK.tor-irc.dnsbl.oftc.net) has joined #ceph
[21:22] * tryggvil_ (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[21:23] * tryggvil_ (~tryggvil@rtr1.tolvusky.sip.is) Quit ()
[21:23] * maelfius (~mdrnstm@ Quit (Quit: Leaving.)
[21:27] * Kioob (~kioob@luuna.daevel.fr) has joined #ceph
[21:29] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) Quit (Ping timeout: 480 seconds)
[21:29] * maelfius (~mdrnstm@ has joined #ceph
[21:35] <dmick> OK, will try a 32-bit VM
[21:36] * MikeMcClurg (~mike@cpc10-cmbg15-2-0-cust205.5-4.cable.virginmedia.com) has joined #ceph
[21:41] * LarsFronius (~LarsFroni@tmo-111-123.customers.d1-online.com) Quit (Quit: LarsFronius)
[22:07] * chutzpah (~chutz@ has joined #ceph
[22:08] * stass (stas@ssh.deglitch.com) Quit (Remote host closed the connection)
[22:14] * stass (stas@ssh.deglitch.com) has joined #ceph
[22:24] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:f4cf:c3b8:f2e7:beb2) has joined #ceph
[22:33] <dmick> build needs libexpat1-dev, not listed in README?...sound right?
[22:33] <dmick> (and libcurl-dev apparently)
[22:34] <dmick> and for the latter I need to choose libcurl4-{openssl,nss,gnutls}-dev
[22:35] <dmick> yehudasa: this is apparently because of --with-radosgw? any wisdom?
[22:36] * EmilienM (~EmilienM@195-132-228-252.rev.numericable.fr) has left #ceph
[22:36] <Tv_> dmick: this is why i hate READMEs duplicating dependency information
[22:36] <yehudasa> dmick: radosgw needs libexpat, libcurl
[22:36] <Tv_> $ grip expat configure.ac
[22:36] <Tv_> configure.ac:191: [AC_CHECK_LIB([expat], [XML_Parse],
[22:36] <Tv_> configure.ac:202: AC_MSG_FAILURE([--with-radosgw was given but libexpat (libexpat1-dev on debian) not found])
[22:37] <dmick> should autoconf run in such a mode as to give me a list of dependencies to install, then?
[22:37] <Tv_> duplicated information means either one or both copies are now wrong
[22:37] <Tv_> dmick: sadly, it doesn't know how to do that, apart from going at it one error at a time
[22:38] <dmick> heh. that would be why it's duplicated then :)
[22:38] <Tv_> a more declarative format would be nice
[22:38] * slang (~slang@207-229-177-80.c3-0.drb-ubr1.chi-drb.il.cable.rcn.com) Quit (Ping timeout: 480 seconds)
[22:38] <Tv_> i'm just saying, the README is routinely wrong
[22:38] <dmick> yeah
[22:38] <dmick> yehudasa: the problem is it apparently needs some flavor of libcurl-dev
[22:38] <dmick> no libcurl available per se
[22:39] <Tv_> dmick: perhaps this is what you want:
[22:39] <Tv_> debian/control:9:Build-Depends: debhelper (>= 6.0.7~), autotools-dev, autoconf, automake, libfuse-dev, libboost-dev (>= 1.34), libedit-dev, libnss3-dev, libtool, libexpat1-dev, libfcgi-dev, libatomic-ops-dev, libgoogle-perftools-dev [i386 amd64], pkg-config, libcurl4-gnutls-dev, libkeyutils-dev, uuid-dev, libaio-dev, python (>= 2.6.6-3~), libxml2-dev
[22:39] <Tv_> that's slightly different from actual upstream build dependencies, but typically good enough to make most users happy
[22:39] <Tv_> also, dpkg-checkbuilddeps
[22:40] <dmick> yyyyyeah
[22:40] <dmick> debian build vs. 'autoconf/make' build makes me nervous, but, at least that's probably a good lead on which libcurl
[22:40] <maelfius> does anyone have any experience at what cluster size (OSD count) more than 3 monitors are required?
[22:40] <Tv_> dmick: background: curl decision is more about licensing than anything else
[22:41] <Tv_> maelfius: if you need to tolerate more than one simultaneous monitor machine failure
[22:41] <Tv_> maelfius: the load requirement to justify >3 is quite unlikely
[22:41] <gregaf> maelfius: very large; I would expect the answer for any real deployment to be 3
[22:42] <maelfius> Tv_: I'm looking at a build with (potentially) ~500 hard drives dedicated to ceph, and according to the docs ideally 1 OSD per drive was the recommendation
[22:42] <maelfius> assuming the network can handle the load that is
[22:42] <Tv_> maelfius: that's, like, a couple of racks of hardware? boooring ;)
[22:43] <maelfius> Tv_: nah, not even a "couple". systems with too many drives in them
[22:43] <Tv_> maelfius: yeah you definitely won't need 5 monitors
[22:43] <Tv_> maelfius: ask again when you go over 500 servers
[22:43] <maelfius> Tv_: figured, but I wanted to 2x check.
[22:43] <gregaf> meh, ask when you hit 5k or your cluster starts breaking in mysterious ways ;)
[22:43] <Tv_> maelfius: and adding mons on the fly is easy too
[22:44] <Tv_> gregaf: hey i want to hear about >500-server deployments
[22:44] <gregaf> well, okay, I like the accounting too, but the monitors don't care at that scale
[22:45] <maelfius> Tv_ / gregaf : aye, will ask and i know adding mons is easy, I just wanted to make sure my current "spec" wasn't outside of a known limit of 3 monitors (I figured they could handle a lot more and the limiting factor would be the network first)
[22:46] * nhorman (~nhorman@2001:470:8:a08:7aac:c0ff:fec2:933b) Quit (Quit: Leaving)
[22:46] <maelfius> thanks again guys, loving working with Ceph! :)
[22:47] * slang (~slang@ace.ops.newdream.net) has joined #ceph
[22:50] <gregaf> let us know how it turns out!
[22:50] <maelfius> gregaf: you know i will! :)
[22:50] * kyle_ (~kyle@ip03.foxyf.simplybits.net) has joined #ceph
[22:54] * rweeks (~rweeks@c-98-234-186-68.hsd1.ca.comcast.net) has joined #ceph
[22:55] <rweeks> hey folks
[22:55] <rweeks> I am running through the quick start for Ceph: http://ceph.com/docs/master/start/quick-start/
[22:55] <rweeks> and encountering weird errors.
[22:57] <joshd> what errors are you seeing?
[22:57] <rweeks> Here is the conf file and the error:
[22:57] <rweeks> http://pastebin.com/N1JZRxqh
[22:59] <Tv_> rweeks: what version of ceph is that?
[22:59] <Tv_> osd_data got a useful default value in 0.46
[23:00] * Kioob (~kioob@luuna.daevel.fr) Quit (Ping timeout: 480 seconds)
[23:00] <Tv_> you probably want to install 0.48.2argonaut before continuing
[23:01] <rweeks> it's whatever the quick-start got me
[23:01] <Tv_> rweeks: dpkg -l ceph
[23:02] <rweeks> 0.41-1ubuntu2.
[23:02] <rweeks> eh
[23:02] <rweeks> relatively old then
[23:02] <Tv_> rweeks: it's almost as if you skipped http://ceph.com/docs/master/start/quick-start/#add-ceph-packages
[23:02] <rweeks> I did go through all that...
[23:02] <Tv_> what does "cat /etc/apt/sources.list.d/ceph.list" say?
[23:03] <rweeks> nothing. but /etc/apt/source has "deb http://ceph.com/debian/ precise main"
[23:03] <Tv_> "/etc/apt/source"? nothing reads that file
[23:04] <rweeks> hmm
[23:04] <Tv_> perhaps the copy-paste failed
[23:04] <pentabular> aren
[23:04] <rweeks> I suspect you are right, Tv_
[23:04] <pentabular> aren't ubuntu packages in the standard repos now?
[23:05] <rweeks> trying again with the copypasta
[23:05] <Tv_> pentabular: ubuntu 12.04 includes v0.41 of ceph, that's ooold
[23:05] <Tv_> pentabular: we're working on getting more stuff into their "cloud archive", but that whole concept is still being defined
[23:06] <Tv_> 12.04 is an LTS -- if you're a paying customer, you get support for everything that was in 12.04 main, including ceph v0.41
[23:07] <Tv_> but for new installs, you really should use argonaut
[23:07] <pentabular> good to know
[23:07] <rweeks> ok
[23:07] <Tv_> and argonaut is much nicer to support, so we'll be happier working that way
[23:07] <kyle_> gregaf: hello, just seeing if you happen to be around. You were helping me with some memory issues yesterday.
[23:07] <rweeks> that got me to 0.48argonaut
[23:07] <rweeks> thank you
[23:08] <gregaf> yep, hi kyle_
[23:08] <rweeks> something must have munged in there first time around
[23:09] <pentabular> Tv_: I'm going to propose some changes to the docs build (well see what you think), but what branch would you prefer to see that on?
[23:09] <kyle_> hello. you mentioned to try going to a "tagged" version is that simply checking out v0.52 for instance?
[23:09] <gregaf> yep
[23:09] * tryggvil (~tryggvil@163-60-19-178.xdsl.simafelagid.is) has joined #ceph
[23:10] <Tv_> pentabular: new branch based on master would be the default answer for new development
[23:10] <rweeks> Tv_: thanks. got my little test cluster working.
[23:10] * tryggvil (~tryggvil@163-60-19-178.xdsl.simafelagid.is) Quit ()
[23:11] <pentabular> Tv_: thanks much! going mad w/ out a 'make clean' in /doc :)
[23:13] <kyle_> is there a reason instruction here at http://ceph.com/wiki/Checking_out do not end with make install, but only make? or should i still be doing make install?
[23:13] <dmick> yay, so 32-bit problem reproduces here.
[23:14] <dmick> kyle_: you may want to install directly, or you may want to set DESTDIR and make a tarball to move around, or you may want to make debs
[23:14] <dmick> (or rpms)
[23:17] * tryggvil (~tryggvil@rtr1.tolvusky.sip.is) has joined #ceph
[23:19] * rosco (~r.nap@ Quit (Quit: *Poof*)
[23:19] * rosco (~r.nap@ has joined #ceph
[23:19] * aliguori (~anthony@ Quit (Remote host closed the connection)
[23:20] <kyle_> gregaf: it looks like using the tagged v0.52 has helped already. When starting osd before, it would immediately begin eat memory until the 16GB was fully consumed. Now it's sitting at < 1GB usage.
[23:21] <gregaf> kyle_: hmm, we must have had a memory leak that got fixed
[23:22] <gregaf> I don't remember hearing about any and didn't see anything obvious when I skimmed the recent commits you'd built off of, but the system is getting bigger than I can track all of ;)
[23:23] <kyle_> no worries. i appreciate you helping me. I just have to get the cluster back online to ensure my large transfer does okay now.
[23:26] <Tv_> pentabular: "make clean" is spelled "rm -rf build-doc" ;)
[23:26] <Tv_> pentabular: (and i much prefer rm -rf over any fancy make tricks)
[23:27] <Tv_> (if it was up to me, all software would build outside of its source dir)
[23:27] <Tv_> kyle_: you most likely want to install packages, not via "make install"
[23:28] <pentabular> well okay, but you're going to quite some trouble to avoid a makefile there
[23:28] <pentabular> there
[23:28] <pentabular> there's a shell script that takes it's place essentially
[23:28] <Tv_> pentabular: and this is worse how?-)
[23:28] <pentabular> why eschew the build tools?
[23:29] <kyle_> yeah i'm not a seasoned vet in that area so that was the best way i knew of to get a new(er) version.
[23:29] <Tv_> pentabular: because the significant chunks are virtualenv and sphinx, neither of which gains anything from make
[23:29] * deepsa (~deepsa@ Quit (Ping timeout: 480 seconds)
[23:29] <Tv_> pentabular: and look at e.g. that command -v dpkg if clause, and ponder about enforcing it with make, with all its $$ lovely escaping
[23:29] <pentabular> sphinx is a standard package now, as is everything you're building into the virtualenv except for asphixiate... (the two custom mods from ceph)
[23:30] <Tv_> pentabular: and sphinx-ditaa
[23:30] <pentabular> that one. :)
[23:30] <pentabular> the actual delta from a standard build and what ceph needs special is teency
[23:30] * LarsFronius (~LarsFroni@2a02:8108:3c0:79:f4cf:c3b8:f2e7:beb2) Quit (Quit: LarsFronius)
[23:31] <pentabular> sphinx produces a nice makefile that can handle both 'man' and 'html' as standard targets
[23:31] <Tv_> pentabular: yeah life is better now that a new enough sphinx is packaged
[23:31] <Tv_> pentabular: i hate the "quickstart" generated thing though :(
[23:32] <pentabular> it's becoming clear that sphinx has maybe grown up a little.
[23:32] <pentabular> hm? re: quickstart?
[23:32] <Tv_> pentabular: sphinx-quickstart is the part generating a makefile, right?
[23:33] <pentabular> oh.. yes, though I was just copying one in to do as little damage as possible
[23:34] <pentabular> the doc build would be faster and smaller w/out downloading & setting up all the python prereqs apart from system packages
[23:34] <Tv_> pentabular: yeah these days you could just insist sphinx is installed
[23:35] <Tv_> pentabular: the two things that are git cloned aren't packaged
[23:35] <pentabular> *and*... although this is just documentation, usually it's wise to go with the platform's packages in case the vendor has patched them in some way (this is the case with some Ubuntu python: M2crypto won't build in a virtualenv)
[23:35] <Tv_> pentabular: that was written on ubuntu 11.10, the sphinx included was too old
[23:35] * tziOm (~bjornar@ti0099a340-dhcp0358.bb.online.no) Quit (Remote host closed the connection)
[23:36] <pentabular> yes, it's pretty clear you had to work-around things like that at the time this was done
[23:36] <pentabular> so.. any interest in a make-over? :)
[23:36] <Tv_> also, i'm less of a fan of "make all your dependencies be packaged" dance as i was 10 years ago
[23:36] <Tv_> it's a lot of effort for not so much gain
[23:37] <Tv_> pentabular: pretty patches accepted
[23:37] <Tv_> frankly, that dance is more needed by C, with the /usr/include /usr/lib conventions so strong -- less so with python & virtualenvs
[23:37] <pentabular> the reqs for docs are an huge delta from the main build, so I wouldn't force it as an automatic target by any means
[23:37] <Tv_> (or go and gopath)
[23:38] <pentabular> indeed... string.h... strings.h...
[23:39] <pentabular> if there's no strong attachment to admin/build-doc I think I'll see what I can do with it.
[23:39] <pentabular> Don't worry: I'm a nice guy. :)
[23:39] <Tv_> pentabular: just beware: target existing doesn't in all cases mean last run executed successfully
[23:39] <Tv_> pentabular: e.g. it runs pip every time on purpose
[23:40] <elder> Tv_> (if it was up to me, all software would build outside of its source dir)
[23:40] <elder> +++
[23:41] <pentabular> a much smaller problem if you use --system-site-packages.
[23:41] <Tv_> pentabular: all in all, i'm kinda over the makefile syntax, and don't see that much value in rewriting admin/build-doc; i'm very much ok with moving sphinx from virtualenv to packaged dependency (and not doing --no-site-packages)
[23:41] <Tv_> pentabular: --no-site-packages often has huge benefits though, when the base os is too old
[23:41] <Tv_> pentabular: as i said, 11.10
[23:41] <pentabular> no prob. understood.
[23:42] <Tv_> pentabular: i guess i should say this explicitly: my fear is we'll go *back* to sphinx in venv as soon as a new upstream release with unicorns and rainbows happens
[23:42] <Tv_> and narwhals, don't forget the narwhals
[23:42] <pentabular> most of what build-doc does is check that prereqs are installed and very little more other than calling sphinx-build
[23:42] * SvenDowideit (~SvenDowid@203-206-171-38.perm.iinet.net.au) has joined #ceph
[23:43] <pentabular> okay, no quickstart makefile then. :) just a nice little tidy one.
[23:44] * pentabular timewarp
[23:44] <pentabular> oh narwhals are so last year!
[23:44] <Tv_> narwhals are mermaid unicorns -- how *could* they go out of fashion?
[23:45] <pentabular> never!
[23:45] <pentabular> ;)
[23:45] <joao> thanks a lot
[23:45] <joao> now I'll have that song stuck in my head for the next 3 days
[23:46] <joao> :(
[23:46] <pentabular> thanks much. I should stop hogging your brainwidth.
[23:46] <pentabular> joao: more cowbell!!
[23:47] <Tv_> you know, whenever a song gets stuck in my head.. i have this amazing skill, i can replace any stuck song with http://www.youtube.com/watch?v=YK3ZP6frAMc and for some reason that doesn't stick
[23:48] <kyle_> sorry to interrupt... i attempted to add an additional monitor yesterday and mistyped the name of the monitor. now my cluster won't respond to things like "ceph health". the logs don't seem to be helping much. Any idea where i can begin looking?
[23:50] <gregaf> how many monitors did you have beforehand?
[23:51] <kyle_> just one. i was attempting to add an additional two but effed up on number two
[23:51] <gregaf> yeah, that's bad
[23:51] <kyle_> thought so
[23:51] <gregaf> your monitor is now refusing to go active because it can't form a quorum (a strict majority of the total it believes to exist)
[23:52] <gregaf> what part of the name did you mistype?
[23:52] <gregaf> or rather, where did you mistype the "name"
[23:52] * jjgalvez1 (~jjgalvez@ Quit (Read error: Connection reset by peer)
[23:53] <kyle_> if i remember correctly it was this step:
[23:53] <kyle_> ceph mon add <name> <ip>[:<port>]
[23:53] <gregaf> okay, so the actual name is an arbitrary thing, if the IP was correct then you can just start up a monitor at that IP with the given name...
[23:58] * deepsa (~deepsa@ has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.