#ceph IRC Log

Index

IRC Log for 2010-11-16

Timestamps are in GMT/BST.

[2:35] * joshd (~joshd@ip-66-33-206-8.dreamhost.com) Quit (Quit: Leaving.)
[2:38] * Jiaju (~jjzhang@222.126.194.154) Quit (Ping timeout: 480 seconds)
[2:42] * sjust (~sam@ip-66-33-206-8.dreamhost.com) Quit (Ping timeout: 480 seconds)
[2:42] * Jiaju (~jjzhang@222.126.194.154) has joined #ceph
[3:01] * Jiaju (~jjzhang@222.126.194.154) Quit (Read error: Connection timed out)
[3:09] * Jiaju (~jjzhang@222.126.194.154) has joined #ceph
[4:10] * cmccabe1 (~cmccabe@dsl081-243-128.sfo1.dsl.speakeasy.net) Quit (Quit: Leaving.)
[4:48] * Jiaju (~jjzhang@222.126.194.154) Quit (Quit: 暂离)
[4:50] * Jiaju (~jjzhang@222.126.194.154) has joined #ceph
[7:53] * f4m8_ is now known as f4m8
[9:05] * allsystemsarego (~allsystem@188.26.32.123) has joined #ceph
[10:04] * Yoric (~David@213.144.210.93) has joined #ceph
[12:10] * Yoric_ (~David@213.144.210.93) has joined #ceph
[12:10] * Yoric (~David@213.144.210.93) Quit (Read error: Connection reset by peer)
[12:10] * Yoric_ is now known as Yoric
[12:14] * Yoric_ (~David@213.144.210.93) has joined #ceph
[12:14] * Yoric (~David@213.144.210.93) Quit (Read error: Connection reset by peer)
[12:14] * Yoric_ is now known as Yoric
[12:15] <jantje> to use 3 active mds do I need to set set_max_mds to 3 or is this on by default?
[12:24] * sentinel_e86 (~sentinel_@188.226.51.71) Quit (Remote host closed the connection)
[12:29] * sentinel_e86 (~sentinel_@188.226.51.71) has joined #ceph
[12:39] <jantje> i have to set it :P
[12:40] <jantje> all mds's are up and running
[12:40] <jantje> although ceph -s can't connect to them I think
[12:40] <jantje> 2010-11-16 12:39:38.046474 7f4af0edc710 -- :/25241 >> 138.203.10.100:6789/0 pipe(0x11b06d0 sd=-1 pgs=0 cs=0 l=0).fault first fault
[12:40] <jantje> 2010-11-16 12:39:41.046540 7f4aedf1a710 -- :/25241 >> 138.203.10.99:6789/0 pipe(0x7f4ae8000af0 sd=-1 pgs=0 cs=0 l=0).fault first fault
[12:40] <jantje> 2010-11-16 12:39:44.046636 7f4aede19710 -- :/25241 >> 138.203.10.100:6789/0 pipe(0x7f4ae8002600 sd=-1 pgs=0 cs=0 l=0).fault first fault
[12:40] <jantje> 2010-11-16 12:39:47.046750 7f4aedd18710 -- :/25241 >> 138.203.10.99:6789/0 pipe(0x7f4ae80020d0 sd=-1 pgs=0 cs=0 l=0).fault first fault
[12:41] <jantje> 2010-11-16 12:39:50.048295 pg v20: 1584 pgs: 1584 active+clean; 26 KB data, 57912 KB used, 1368 GB / 1380 GB avail
[12:41] <jantje> 2010-11-16 12:39:50.050596 mds e12: 3/3/3 up {0=up:active,1=up:active,2=up:active}
[12:41] <jantje> 2010-11-16 12:39:50.050622 osd e9: 6 osds: 6 up, 6 in
[12:41] <jantje> 2010-11-16 12:39:50.050663 log 2010-11-16 12:39:31.023434 mon0 138.203.10.98:6789/0 14 : [INF] mds2 138.203.10.100:6800/6822 up:active
[12:41] <jantje> 2010-11-16 12:39:50.050696 mon e1: 3 mons at {0=138.203.10.98:6789/0,1=138.203.10.99:7890/0,2=138.203.10.100:8901/0}
[12:41] <jantje> could it be related to the 'multiple' interface problem someone talked about on the mailinglist?
[12:41] <jantje> I have found bonded interfaces
[14:29] <jantje> sagewk: I can still produce #549, even with 2.6.37-rc1+ on my client
[14:30] <jantje> please let me know what debugging info you need from me
[14:45] <jantje> sagewk: I have each mon running on a different port, I think i it using the port of the first MON for all other MONs
[14:50] <jantje> (and yes, I did a new mkcephfs, so it re-created the mon map)
[14:52] <jantje> I changed the mon ports back, and I can use my secondary and third MON's again when mounting on the client
[15:00] <jantje> And I would like to use the configuration file to set_max_mds to 3, it works withthe command line, but not with the config.. :)
[15:00] <jantje> sorry for all those questions :-)
[16:14] <jantje> 2010-11-16 16:13:53.985020 7fa6d757d710 mds1.migrator finish_import_inode_caps for client4113 on [inode 1000000169a [...2,head] ~mds1/stray/1000000169a auth{0=1,2=1} v44867 snaprealm=0xc2a6900 s=174710 nl=0 n(v0 b174710 1=1+0) (iversion lock) | importingcaps replicated dirty 0x670bfa0]
[16:14] <jantje> is this something from the multi-mds stuff ?
[16:16] <jantje> rm: WARNING: Circular directory structure.
[16:16] <jantje> This almost certainly means that you have a corrupted file system.
[16:16] <jantje> interesting!
[16:17] <jantje> maybe :P
[16:53] * DLange (~DLange@dlange.user.oftc.net) Quit (Quit: reboot)
[16:56] * DLange (~DLange@dlange.user.oftc.net) has joined #ceph
[17:02] * greglap (~Adium@cpe-76-90-74-194.socal.res.rr.com) has joined #ceph
[17:02] * greglap (~Adium@cpe-76-90-74-194.socal.res.rr.com) Quit (Quit: Leaving.)
[18:32] * Yoric (~David@213.144.210.93) Quit (Quit: Yoric)
[18:55] * greglap (~Adium@166.205.139.165) has joined #ceph
[18:56] * joshd (~joshd@ip-66-33-206-8.dreamhost.com) has joined #ceph
[19:04] <sagewk> jantje: interesting. what does 'ceph mon stat' say?
[19:05] <sagewk> jantje: er, actually, what was the mount command you gave? you can specify multiple montior ips by separating with commas, like
[19:05] <sagewk> 'mount -t ceph mon1,mnon2,mon3:/ mnt'
[19:05] <sagewk> or add :port to each of those. it may be that the parsing for the :port part is broken, though
[19:06] <sagewk> that message is harmless, btw; i'll fix the debug level
[19:15] * greglap1 (~Adium@166.205.137.142) has joined #ceph
[19:19] * sjust (~sam@ip-66-33-206-8.dreamhost.com) has joined #ceph
[19:22] * greglap (~Adium@166.205.139.165) Quit (Ping timeout: 480 seconds)
[20:07] * Yoric (~David@dau94-10-88-189-211-192.fbx.proxad.net) has joined #ceph
[20:17] * greglap1 (~Adium@166.205.137.142) Quit (Ping timeout: 480 seconds)
[20:31] * greglap (~Adium@ip-66-33-206-8.dreamhost.com) has joined #ceph
[20:59] * MarkN (~nathan@59.167.240.178) Quit (Ping timeout: 480 seconds)
[21:08] * MarkN (~nathan@59.167.240.178) has joined #ceph
[22:31] * vituko (~vituko@76.180.18.95.dynamic.jazztel.es) Quit (Quit: Saliendo)
[22:42] * johnl (~johnl@cpc3-brad19-2-0-cust563.barn.cable.virginmedia.com) has joined #ceph
[22:58] <jantje> sagewk: I just used a single ip, but I couldn't connect to the others
[22:58] <jantje> (on the client to mount)
[22:58] <sagewk> kernel client?
[22:58] <jantje> yes
[22:59] <jantje> .37rc1
[22:59] <sagewk> the client uses the ip(s) you specify to make an initial connection.. once that happens it ges the full list of monitors. so specifying multiple ips is important just for the mount step to be reliable
[23:06] <jantje> I agree, but it should be able to use the ip of the second or third mon to make the initial connection
[23:07] <jantje> and that didn't work
[23:07] <sagewk> oh i see. any ip should work.
[23:09] <sagewk> and seems to work for me. can you reproduce the problem?
[23:13] <jantje> have you tried chaning the mon ports?
[23:13] <jantje> *changing
[23:14] <gregaf> jantje: are you specifying the mon ports in your mount command?
[23:16] <jantje> well well, no that's not really the issue I think, I didn't try to mount the filesystem using the second or third MON
[23:16] <jantje> ceph -s
[23:16] <jantje> 2010-11-16 12:39:38.046474 7f4af0edc710 -- :/25241 >> 138.203.10.100:6789/0 pipe(0x11b06d0 sd=-1 pgs=0 cs=0 l=0).fault first fault
[23:16] <jantje> 12:40 < jantje> 2010-11-16 12:39:41.046540 7f4aedf1a710 -- :/25241 >> 138.203.10.99:6789/0 pipe(0x7f4ae8000af0 sd=-1 pgs=0 cs=0 l=0).fault first f
[23:16] <jantje> is using the wrong ports
[23:16] <jantje> (I think...)
[23:18] <jantje> port for :99 shouold be 7890 and .100 should be 8901 or something like that
[23:19] <gregaf> what's the command you're running to get that?
[23:19] <gregaf> are you specifying them on the command-line or via the ceph.conf?
[23:27] * Yoric (~David@dau94-10-88-189-211-192.fbx.proxad.net) Quit (Quit: Yoric)
[23:34] <jantje> ceph.conf
[23:35] <gregaf> and the monitor ports are specified correctly in the config file?
[23:35] <jantje> mon addr = 138.203.10.98:6789
[23:35] <jantje> and for each mon I changed the ports
[23:36] <jantje> mon0 had 6789, mon1 7890, mon2 ...
[23:36] <gregaf> it's possible that we aren't parsing out ports correctly out of the conf file, although I'd really expect that it's just getting the wrong data from somewhere
[23:37] <gregaf> do you have incorrect config files located elsewhere in the search areas, maybe?
[23:38] <jantje> Hmm, I still had an old one in /etc/ceph/
[23:38] <jantje> but I was expecting it to take ./ceph.conf
[23:38] <jantje> but you could be right
[23:38] <gregaf> yeah, I think the local dir is the last place it looks
[23:39] <jantje> whats the correct ceph.conf syntax to use 3 mds's ?
[23:39] <jantje> mds max mds = 3
[23:39] <jantje> or mds set_max_mds = 3
[23:39] <jantje> (I think I tried some possibilities, but it probably took the wrong config file as well ! :)
[23:44] <jantje> Sometimes I really hate it to bother you guys with this :)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.