#ceph IRC Log

Index

IRC Log for 2012-08-05

Timestamps are in GMT/BST.

[0:06] * MarkDude (~MT@c-98-210-253-235.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[1:15] * BManojlovic (~steki@212.200.241.176) Quit (Quit: Ja odoh a vi sta 'ocete...)
[1:16] * tnt (~tnt@167.39-67-87.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[1:16] * themgt (~themgt@24-181-215-214.dhcp.hckr.nc.charter.com) Quit (Quit: themgt)
[1:21] * guerby (~guerby@nc10d-ipv6.tetaneutral.net) Quit (Remote host closed the connection)
[1:26] * guerby (~guerby@nc10d-ipv6.tetaneutral.net) has joined #ceph
[2:00] * tjikkun (~tjikkun@2001:7b8:356:0:225:22ff:fed2:9f1f) Quit (Remote host closed the connection)
[2:12] * themgt (~themgt@24-181-215-214.dhcp.hckr.nc.charter.com) has joined #ceph
[2:15] * themgt (~themgt@24-181-215-214.dhcp.hckr.nc.charter.com) Quit ()
[2:30] * joshd (~jdurgin@2602:306:c5db:310:1e6f:65ff:feaa:beb7) has joined #ceph
[2:30] * joshd (~jdurgin@2602:306:c5db:310:1e6f:65ff:feaa:beb7) Quit ()
[2:57] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Quit: Leseb)
[4:24] * bshah (~bshah@sproxy2.fna.fujitsu.com) Quit (Server closed connection)
[5:32] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:41] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[6:45] * Cube (~Adium@cpe-76-95-223-199.socal.res.rr.com) has joined #ceph
[6:49] * cattelan (~cattelan@2001:4978:267:0:21c:c0ff:febf:814b) Quit (Ping timeout: 480 seconds)
[8:53] * glowell (~glowell@c-98-210-226-131.hsd1.ca.comcast.net) has joined #ceph
[8:53] * glowell (~glowell@c-98-210-226-131.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[10:08] * tnt (~tnt@167.39-67-87.adsl-dyn.isp.belgacom.be) has joined #ceph
[10:31] * EmilienM (~EmilienM@191.223.101.84.rev.sfr.net) has joined #ceph
[10:58] * alexxy (~alexxy@2001:470:1f14:106::2) Quit (Ping timeout: 480 seconds)
[11:27] * BManojlovic (~steki@212.200.241.176) has joined #ceph
[11:45] * BManojlovic (~steki@212.200.241.176) Quit (Quit: Ja odoh a vi sta 'ocete...)
[12:20] * Cube (~Adium@cpe-76-95-223-199.socal.res.rr.com) Quit (Quit: Leaving.)
[12:30] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) has joined #ceph
[13:45] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[14:04] * EmilienM (~EmilienM@191.223.101.84.rev.sfr.net) Quit (Quit: Leaving...)
[15:13] * asadpanda (~asadpanda@2001:470:c09d:0:20c:29ff:fe4e:a66) Quit (Ping timeout: 480 seconds)
[15:34] * vhasi (martin@vha.si) Quit (Read error: Connection reset by peer)
[15:34] * vhasi (martin@vha.si) has joined #ceph
[15:37] <exec> tnt: ping?
[15:38] <tnt> pong
[15:38] <exec> wow )
[15:38] <exec> hi
[15:39] <tnt> hi
[15:39] <exec> could you please comment my setup? (I'll post details r. now)
[15:39] <exec> I still have no stability on my cluster.
[15:41] <exec> main idea - share somehow disks from 5 servers. these servers used as backup storage. I can use rbd images mounted to one of them in one time. or use cephfs.
[15:42] <exec> however, documentation says that rbd shouln't used on the samve server with osd. is it true for now?
[15:43] <tnt> huh ... I'm not exactly an expert ...
[15:43] <exec> however, has you got stable cluster somewhere? )
[15:43] <exec> have you got )
[15:43] <exec> sorry for my english )
[15:44] <tnt> not yet. I'm in the process of evaluationg ceph right now.
[15:44] <exec> heh. I'm on the same place.
[15:44] <exec> ok. can you say smth about one osd per server vs one osd per disk?
[15:45] <tnt> I'm planning to use one osd per disk
[15:45] <exec> I have WARN in logs that it can be slow.
[15:45] <exec> me too
[15:45] <exec> do you plan to use cephfs or rdb images only?
[15:46] <tnt> rbd and radosgw, no cephfs for now. AFAIK it's not production ready so can't use it yet.
[15:46] <exec> and what filesystem for osd storage
[15:46] <exec> ?
[15:46] <tnt> xfs
[15:46] <exec> hm.
[15:47] <exec> I've tried many options, for now it's ext4 with rbd.
[15:48] <exec> I can create big image (1TB), map it and test direct speed via hdparm/dd
[15:48] <exec> everything looks ok
[15:50] <tnt> what's the problem then ?
[15:50] <exec> however, when I make ext4 on rbd, mount it and "dd if=/dev/zero bs=1M count=10000 of=/mnt/test/t.img", all osd start writes, then stop and looks like down from monitors.
[15:50] <exec> small amount of write pass
[15:52] <exec> looks like there are some deadlock, b/c ceph-osd processes stay as zombies
[15:54] <exec> can you reproduce this?
[15:54] <exec> if you have some time ofc.
[15:56] <tnt> I never had the problem but I'm running on xfs not ext4
[15:56] <exec> I've got the same thing with xfs too. will try again.
[15:57] <exec> btw, what is you kernel version?
[15:57] <exec> and have you map/use rbd on the same server with OSDs?
[15:57] <tnt> 3.2 IIRC ... it's an ubuntu precise 12.04
[15:58] <tnt> I won't map rbd on the same server as I run the osd in productions. But I did that during one of the test and copied large files and it worked fine.
[15:58] <exec> ok. the same as mine
[15:59] <exec> in my setup rbd should be mapped on the same servers with osds
[16:00] <exec> ok. I'll try other options here. do you have any issues at all?
[16:03] <exec> I've even wrote fabric task, so I can recreate cluster from scratch in a two minutes )
[16:50] <joao> <exec> however, when I make ext4 on rbd, mount it and "dd if=/dev/zero bs=1M count=10000 of=/mnt/test/t.img", all osd start writes, then stop and looks like down from monitors.
[16:50] <joao> this is exactly something the mailing list would be interested in hearing about
[16:50] <lxo> wow, my ceph cluster is *flying* after converting all osds to 64k-metadata-blocks btrfs!
[16:51] <joao> cool :)
[17:25] * lofejndif (~lsqavnbok@09GAAG6RO.tor-irc.dnsbl.oftc.net) has joined #ceph
[18:00] * jeffhung_ (~jeffhung@60-250-103-120.HINET-IP.hinet.net) has joined #ceph
[18:00] * MapspaM (~clint@xencbyrum2.srihosting.com) has joined #ceph
[18:00] * jeffhung (~jeffhung@60-250-103-120.HINET-IP.hinet.net) Quit (Read error: Connection reset by peer)
[18:02] * benner_ (~benner@193.200.124.63) has joined #ceph
[18:03] * guerby_ (~guerby@nc10d-ipv6.tetaneutral.net) has joined #ceph
[18:04] * SpamapS (~clint@xencbyrum2.srihosting.com) Quit (Ping timeout: 480 seconds)
[18:04] * benner (~benner@193.200.124.63) Quit (Read error: Connection reset by peer)
[18:04] * guerby (~guerby@nc10d-ipv6.tetaneutral.net) Quit (Ping timeout: 480 seconds)
[18:06] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[18:08] * cattelan (~cattelan@2001:4978:267:0:21c:c0ff:febf:814b) has joined #ceph
[18:17] * loicd1 (~loic@brln-4dba8516.pool.mediaWays.net) has joined #ceph
[18:21] * loicd (~loic@brln-d9ba4102.pool.mediaWays.net) Quit (Ping timeout: 480 seconds)
[18:56] <exec> joao: will try later. I need more details about it
[19:04] * BManojlovic (~steki@212.200.241.176) has joined #ceph
[19:05] * lofejndif (~lsqavnbok@09GAAG6RO.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)
[19:29] <lxo> hey, look, there's been a 0.49 release out for a while; I'd completely missed it. is 0.50 expected soon? (then I might as well wait a bit longer on 0.48)
[19:49] <exec> from: http://ceph.com/wiki/Rbd "Warning: Don't use rbd kernel driver on the osd server. Perhaps it will freeze the rbd client and your osd server." - it's my issue. any chances to use osd and rbd module on the same server?
[19:51] <exec> lxo: why don't stay on "long-term support" 0.48? team promises to backport new cool features into that
[19:52] <lxo> exec, well, I guess I could, but then I'd be stuck with old features, and my upcoming patches might not be as useful
[19:53] <exec> then it's up2you. I mean production usage )
[19:55] <lxo> exec, yeah, mine is just a home cluster I'm using mainly for redundant storage, so far mainly of backups
[19:55] * EmilienM (~EmilienM@191.223.101.84.rev.sfr.net) has joined #ceph
[19:59] * allsystemsarego (~allsystem@188.27.167.57) has joined #ceph
[20:51] * EmilienM (~EmilienM@191.223.101.84.rev.sfr.net) has left #ceph
[21:19] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Quit: Leseb)
[21:22] * Ryan_Lane (~Adium@c-67-160-217-184.hsd1.ca.comcast.net) has joined #ceph
[21:55] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) has joined #ceph
[21:57] <iggy> exec: kernel drivers talking to userspace on the same server all have that issue... it's not specific to rbd, and there's not really a good way around it
[21:58] * Leseb_ (~Leseb@62.233.37.47) has joined #ceph
[22:03] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Ping timeout: 480 seconds)
[22:03] * Leseb_ is now known as Leseb
[22:37] * Leseb (~Leseb@62.233.37.47) Quit (Ping timeout: 480 seconds)
[22:40] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) has joined #ceph
[22:59] <exec> iggy: so, we'll have this issue with cephfs? btw, what is the status of it?
[23:00] <iggy> exec: yes, and samba, and every other kernel driver that talks to userspace
[23:02] <iggy> there are (dirty) ways to mitigate it, but it's a tricky situation
[23:02] <exec> hm. I have N servers with limited traffic. and all of them should be used for remote backups.
[23:02] <exec> I can use on server as main point of share, but it limited for bandwidth...
[23:02] * wikiency (~wikiency@178.124.160.246) has joined #ceph
[23:03] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Read error: Connection reset by peer)
[23:03] <exec> perhaps, I'll use some proxy techinic there..
[23:03] * wikiency (~wikiency@178.124.160.246) has left #ceph
[23:04] <exec> if there are no possibility of common storage usage between osd nodes..
[23:04] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) has joined #ceph
[23:04] <iggy> you may not ever have a problem, but the one time you do...
[23:05] <iggy> you can use fuse or librbd without worry
[23:05] <exec> I have the porblem every time with rbd formatted and mounted. no differences with FS on top of it and as backend fs for osd...
[23:05] <exec> librbd - via api? )
[23:06] <iggy> you can use it multiple ways... qemu uses it directly
[23:06] <iggy> for instance
[23:07] <exec> I thought about it
[23:08] <exec> use one VM on each osd server.
[23:08] * mikeryan (mikeryan@lacklustre.net) Quit (Remote host closed the connection)
[23:08] * mikeryan (mikeryan@lacklustre.net) has joined #ceph
[23:08] <exec> but some proxying looks better )
[23:09] <iggy> one vm on each?
[23:10] <exec> yup. b/c/ each server has its own traffic limit
[23:11] <iggy> oh... some sort of hosting thing i guess
[23:11] <exec> aha )
[23:12] <exec> my backup works via rsync + hardlink solutions. customized somehow...
[23:12] <iggy> tricky situation
[23:12] <exec> seems like I can used rsync via web or ssh proxy )
[23:13] <exec> situation likw this gorce brainstorm )
[23:13] <exec> bre... force )
[23:13] <exec> brrr.
[23:13] <exec> I've drank some wine. sorry )
[23:14] * mikeryan (mikeryan@lacklustre.net) Quit (Remote host closed the connection)
[23:15] <exec> cephfs works much better. vs rbd images on the same server. I've enables sync to it. will look at tme morning how it works..
[23:24] * mikeryan (mikeryan@lacklustre.net) has joined #ceph
[23:32] * mikeryan (mikeryan@lacklustre.net) Quit (Remote host closed the connection)
[23:36] * tnt (~tnt@167.39-67-87.adsl-dyn.isp.belgacom.be) Quit (Ping timeout: 480 seconds)
[23:41] * mikeryan (mikeryan@lacklustre.net) has joined #ceph
[23:46] * Leseb_ (~Leseb@62.233.37.59) has joined #ceph
[23:52] * Leseb (~Leseb@5ED01FAC.cm-7-1a.dynamic.ziggo.nl) Quit (Ping timeout: 480 seconds)
[23:52] * Leseb_ is now known as Leseb

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.