#ceph IRC Log

Index

IRC Log for 2013-11-23

Timestamps are in GMT/BST.

[0:00] * xarses (~andreww@204.11.231.50.static.etheric.net) has joined #ceph
[0:01] * sleinen1 (~Adium@2001:620:0:26:1167:edbb:fd1:6af) Quit (Quit: Leaving.)
[0:01] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Ping timeout: 480 seconds)
[0:02] * rendar (~s@host254-179-dynamic.1-87-r.retail.telecomitalia.it) Quit ()
[0:03] * dmsimard (~Adium@70.38.0.248) Quit (Quit: Leaving.)
[0:03] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[0:06] * gregmark (~Adium@cet-nat-254.ndceast.pa.bo.comcast.net) Quit (Quit: Leaving.)
[0:06] * xmltok (~xmltok@216.103.134.250) has joined #ceph
[0:07] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[0:13] * sarob_ (~sarob@ip-64-134-225-149.public.wayport.net) Quit (Remote host closed the connection)
[0:14] * mtanski (~mtanski@69.193.178.202) Quit (Ping timeout: 480 seconds)
[0:16] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[0:16] * ChanServ sets mode +o joao|lap
[0:17] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[0:22] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[0:23] * fireD_ (~fireD@93-139-168-145.adsl.net.t-com.hr) Quit (Quit: leaving)
[0:25] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[0:26] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Ping timeout: 480 seconds)
[0:27] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:27] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[0:27] * ChanServ sets mode +o joao|lap
[0:27] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) Quit (Quit: Ex-Chat)
[0:28] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Read error: Operation timed out)
[0:28] * odyssey4me (~odyssey4m@41.76.192.48) has joined #ceph
[0:30] * scuttlemonkey changes topic to 'CDS Firefly Schedule available! http://goo.gl/LOhq3O || Latest stable (v0.72.0 "Emperor") -- http://ceph.com/get || dev channel #ceph-devel || Ceph Blog -- http://ceph.com/community/blog/'
[0:32] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[0:32] * ChanServ sets mode +v andreask
[0:36] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Ping timeout: 480 seconds)
[0:40] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[0:41] * andreask (~andreask@zid-vpnn096.uibk.ac.at) has joined #ceph
[0:41] * ChanServ sets mode +v andreask
[0:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:44] * yanzheng (~zhyan@jfdmzpr02-ext.jf.intel.com) has joined #ceph
[0:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:54] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Quit: ...)
[0:54] * yanzheng (~zhyan@jfdmzpr02-ext.jf.intel.com) Quit (Remote host closed the connection)
[0:54] * odyssey4me (~odyssey4m@41.76.192.48) Quit (Read error: Connection reset by peer)
[0:58] * bandrus (~Adium@107.216.171.224) Quit (Quit: Leaving.)
[1:02] * bandrus (~Adium@107.216.171.224) has joined #ceph
[1:08] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Remote host closed the connection)
[1:11] * sarob (~sarob@216.70.157.60) has joined #ceph
[1:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[1:16] * xarses (~andreww@204.11.231.50.static.etheric.net) Quit (Ping timeout: 480 seconds)
[1:17] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[1:18] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:21] * ircolle (~Adium@2601:1:8380:2d9:8d67:a3fe:f259:cc67) Quit (Quit: Leaving.)
[1:27] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[1:27] * bandrus1 (~Adium@107.216.171.224) has joined #ceph
[1:29] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[1:30] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[1:34] * bandrus (~Adium@107.216.171.224) Quit (Ping timeout: 480 seconds)
[1:37] * bandrus (~Adium@107.216.171.224) has joined #ceph
[1:37] * tsnider (~tsnider@198.95.226.40) Quit (Quit: Leaving.)
[1:40] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[1:41] * bandrus1 (~Adium@107.216.171.224) Quit (Ping timeout: 480 seconds)
[1:41] * alram (~alram@38.122.20.226) Quit (Quit: leaving)
[1:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[1:45] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:46] * xmltok (~xmltok@216.103.134.250) Quit (Quit: Leaving...)
[1:50] * KindOne (KindOne@0001a7db.user.oftc.net) has joined #ceph
[1:54] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) Quit (Quit: Leaving.)
[1:55] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:56] * davidzlap (~Adium@cpe-23-242-31-175.socal.res.rr.com) has joined #ceph
[1:56] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[2:05] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) Quit (Quit: Computer has gone to sleep.)
[2:05] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[2:06] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[2:10] * sjusthm (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Remote host closed the connection)
[2:12] * bandrus (~Adium@107.216.171.224) Quit (Ping timeout: 480 seconds)
[2:20] * bandrus (~Adium@108.249.93.104) has joined #ceph
[2:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:21] * diegows (~diegows@190.190.11.42) has joined #ceph
[2:21] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[2:26] * aliguori (~anthony@74.202.210.82) Quit (Remote host closed the connection)
[2:27] * dlan (~dennis@116.228.88.131) Quit (Remote host closed the connection)
[2:28] * dlan (~dennis@116.228.88.131) has joined #ceph
[2:30] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[2:30] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[2:32] * bandrus1 (~Adium@c-98-238-148-252.hsd1.ca.comcast.net) has joined #ceph
[2:32] * bandrus (~Adium@108.249.93.104) Quit (Ping timeout: 480 seconds)
[2:33] * bandrus (~Adium@107.216.174.246) has joined #ceph
[2:35] * linuxkidd (~linuxkidd@ip72-193-217-254.lv.lv.cox.net) Quit (Quit: Konversation terminated!)
[2:40] * bandrus1 (~Adium@c-98-238-148-252.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[2:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[2:44] * angdraug (~angdraug@64-79-127-122.static.wiline.com) Quit (Quit: Leaving)
[2:46] * e1mer (~erivera@121.54.77.87) has joined #ceph
[2:48] <mrjack_> https://www.smart-weblications.de/images/kvm_io_performance.png
[2:48] <mrjack_> :)
[2:51] * peetaur is now known as Guest6664
[2:51] * peetaur (~peter@CPE788df73fb301-CM788df73fb300.cpe.net.cable.rogers.com) has joined #ceph
[2:53] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:55] <nigwil> impressive mrjack :-)
[2:55] * KindTwo (KindOne@198.14.198.2) has joined #ceph
[2:56] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[2:56] * KindTwo is now known as KindOne
[2:56] * Guest6664 (~peter@CPE788df73fb301-CM788df73fb300.cpe.net.cable.rogers.com) Quit (Ping timeout: 480 seconds)
[3:00] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:01] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[3:01] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) Quit (Quit: Leaving.)
[3:02] * KevinPerks1 (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[3:02] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Read error: Connection reset by peer)
[3:09] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[3:13] * KevinPerks1 (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[3:15] * Cube (~Cube@12.248.40.138) Quit (Ping timeout: 480 seconds)
[3:20] <nwf> There's not, by chance, a ceph-osd fsck or other tool that attempts to rescue objects from a damaged OSD store? :(
[3:21] * sarob (~sarob@216.70.157.60) Quit (Remote host closed the connection)
[3:21] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[3:21] * sarob (~sarob@216.70.157.60) has joined #ceph
[3:22] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[3:22] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Read error: Connection reset by peer)
[3:22] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[3:24] * andreask (~andreask@zid-vpnn096.uibk.ac.at) Quit (Read error: Connection reset by peer)
[3:25] * diegows (~diegows@190.190.11.42) Quit (Ping timeout: 480 seconds)
[3:25] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:29] * sarob (~sarob@216.70.157.60) Quit (Ping timeout: 480 seconds)
[3:30] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[3:35] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[3:41] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[3:47] * e1mer (~erivera@121.54.77.87) Quit (Quit: This computer has gone to sleep)
[3:49] * Pedras (~Adium@216.207.42.132) Quit (Ping timeout: 480 seconds)
[3:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:54] <dmick> nwf: well, there is osd repair and pg repair, but, I would read about them before trying them
[3:58] * Sysadmin88 (~IceChat77@94.1.37.151) has joined #ceph
[3:59] <nwf> dmick: Well, the problem we have is that the OSDs are so damaged that they crash rather than staying up long enough to interact with the cluster.
[4:00] <nwf> Our underlying filesystem (ZoL) really shit the bed and many of the xattrs are gone or unreadable.
[4:00] <dmick> oh dear
[4:00] <nwf> Yes. I get to spend tomorrow leveling our machines and starting over.
[4:01] <nwf> Unfortunately this counts as "production" for us, but we're a student group, so it's relatively minor. :)
[4:01] * yanzheng (~zhyan@134.134.139.74) has joined #ceph
[4:01] <dmick> well, there are some filestore manipulation tools in the source tree, but I don't know them well or if they might help
[4:01] <dmick> they're mostly expert tools AFAIK
[4:07] <nwf> I'll look. I'd really like to have some hope of rescuing our VM images from ceph, but I don't think there's much chance.
[4:07] <nwf> (This was not the most carefully done setup, so we have a lot of things in ceph that are not backed up otherwise. Blah!)
[4:08] <dmick> what happened to the zfs?
[4:08] <nwf> I stupidly "zfs set xattr=sa" on a bunch of OSDs' backing stores. This was a mistake, despite that the machine that has been using that configuration from the beginning is fine.
[4:08] <nwf> It was supposed to help speed, as the sa form of xattrs is less-seeky than the default, but is linux-specific.
[4:09] <nwf> The Linux-specificity doesn't really matter because they're just being used for OSDs, so...
[4:09] <nwf> But now we have pools that return EFAULT to getfattr, that simply don't have any xattrs at all, or that panic the kernel on scrub.
[4:09] <nwf> Needless to say Ceph is very unhappy with us.
[4:10] <dmick> is your theory that the sa xattrs are just buggy?
[4:10] <nwf> One of the ZFS nodes -- the one already with the xattr=sa knob set -- and the worker using dmraid+btrfs are both up, but unfortunately basically all our pools straddled some OSD that's just found greener pastures.
[4:10] <dmick> they look like the right setting
[4:11] <nwf> No, I think it's the attempt to use both types at once that's buggy.
[4:11] <dmick> i.e. the way xattrs ought to be stored
[4:11] <nwf> Like I said, we have a machine with xattr=sa set from the very beginning that's been fine.
[4:11] <dmick> oh, you mean setting the pool after it already had data
[4:11] <nwf> But I think you're taking your life into your hand if you change that knob after there are files in the dataset.
[4:11] <nwf> Yes.
[4:11] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[4:11] <dmick> i c
[4:12] <nwf> Live and learn, I just wish I hadn't nuked a month's of work.
[4:13] <dmick> one might think that, if it were a known problem, zfs set would reject that; I wonder if it's really a bug
[4:13] <dmick> but I don't know ZoL really at all
[4:13] <dmick> ISTR other settings that didn't affect older data but would affect new files (like dedup, and block size, and stuff)
[4:13] <nwf> Well, I mean to stress test it later and produce a more minimized test case than "run Ceph on four X4500s, make this change, and wait overnight for your life to become sad", but getting a working infrastructure is more important at the moment. :)
[4:14] <dmick> that is, effectively applied incrementally on next write
[4:14] <nwf> Yeah, ZFS is supposed to be CoW like that.
[4:15] <nwf> My experience with ZFS has not always been the most pleasant thing, tho'. The assumption that "If it has a valid checksum then it must pass all other data invariants" is a great way for bugs to induce pool-wide loss.
[4:16] <nwf> http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016627.html , for example, on a separate system (FreeBSD ZFS, but the ZFS part is what matters).
[4:17] * dpippenger (~riven@cpe-76-166-208-83.socal.res.rr.com) has joined #ceph
[4:22] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[4:24] <dmick> yeah...
[4:27] * Cube (~Cube@netblock-72-25-110-178.dslextreme.com) has joined #ceph
[4:30] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:30] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[11:17] -oxygen.oftc.net- *** Looking up your hostname...
[11:17] -oxygen.oftc.net- *** Checking Ident
[11:17] -oxygen.oftc.net- *** Couldn't look up your hostname
[11:17] -oxygen.oftc.net- *** No Ident response
[11:17] * CephLogBot (~PircBot@92.63.168.213) has joined #ceph
[11:17] * Topic is 'CDS Firefly Schedule available! http://goo.gl/LOhq3O || Latest stable (v0.72.0 "Emperor") -- http://ceph.com/get || dev channel #ceph-devel || Ceph Blog -- http://ceph.com/community/blog/'
[11:17] * Set by scuttlemonkey!~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net on Sat Nov 23 00:30:08 CET 2013
[11:20] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[11:32] -sig_wall- don't strain, NSA is already logging
[11:40] * odyssey4me (~odyssey4m@41.76.192.48) Quit (Quit: odyssey4me)
[11:41] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:44] * KindTwo (KindOne@50.96.229.7) has joined #ceph
[11:47] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[11:47] * KindTwo is now known as KindOne
[11:48] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[11:48] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[11:49] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[11:57] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:07] * e1mer (~erivera@121.54.77.87) has joined #ceph
[12:11] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[12:16] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[12:16] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit ()
[12:19] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) Quit (Quit: Computer has gone to sleep.)
[12:23] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:23] * BillK (~BillK-OFT@124-148-85-36.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[12:25] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Quit: ZNC - http://znc.in)
[12:25] * BillK (~BillK-OFT@220-253-138-225.dyn.iinet.net.au) has joined #ceph
[12:26] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[12:32] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[12:34] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[12:37] * xdeller_ (~xdeller@95-31-29-125.broadband.corbina.ru) has joined #ceph
[12:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[12:44] * simulx (~simulx@66-194-114-178.static.twtelecom.net) has joined #ceph
[12:47] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Quit: ZNC - http://znc.in)
[12:47] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[12:48] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit ()
[12:48] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[12:48] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:05] * diegows (~diegows@190.190.11.42) has joined #ceph
[13:07] * KindTwo (KindOne@h28.38.186.173.dynamic.ip.windstream.net) has joined #ceph
[13:08] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[13:08] * KindTwo is now known as KindOne
[13:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:14] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) has joined #ceph
[13:15] * diegows (~diegows@190.190.11.42) Quit (Quit: Leaving)
[13:15] * diegows (~diegows@190.190.11.42) has joined #ceph
[13:17] * BillK (~BillK-OFT@220-253-138-225.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[13:18] <Sodo> sjustwork, hi
[13:19] * BillK (~BillK-OFT@106-68-16-238.dyn.iinet.net.au) has joined #ceph
[13:26] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[13:26] <loicd> madkiss: we had interesting discussions @c-base to use ceph to store the music collection. The disk array died and was to be replaced by something.
[13:28] <madkiss> that sounds interesting :)
[13:29] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:31] * BillK (~BillK-OFT@106-68-16-238.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[13:32] * BillK (~BillK-OFT@124-169-74-6.dyn.iinet.net.au) has joined #ceph
[13:34] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[13:41] <loicd> so to speak :-D
[13:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:47] <loicd> does someone around here uses ceph with IPv6 ?
[13:48] * madkiss (~madkiss@213162068091.public.t-mobile.at) Quit (Quit: Leaving.)
[13:50] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) Quit (Quit: Computer has gone to sleep.)
[13:53] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[13:54] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:55] <joao> loicd, wido does iirc
[13:55] <joao> he mentioned some issues with that
[13:56] <loicd> ah, interesting
[13:56] <loicd> btw wido: how was the cloudstack conference ?
[13:56] <joao> he was even working on a ticket on the tracker regarding, iirc, something about bind taking just long enough to bind to the ipv6 nic and something weird happening
[13:56] * BillK (~BillK-OFT@124-169-74-6.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[13:57] * BillK (~BillK-OFT@58-7-73-113.dyn.iinet.net.au) has joined #ceph
[14:02] * danieagle (~Daniel@186.214.73.228) has joined #ceph
[14:04] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[14:05] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[14:06] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit ()
[14:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:12] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[14:13] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[14:13] * diegows (~diegows@190.190.11.42) Quit (Ping timeout: 480 seconds)
[14:13] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) Quit ()
[14:14] * e1mer (~erivera@121.54.77.87) Quit (Quit: This computer has gone to sleep)
[14:14] * rongze (~rongze@211.155.113.166) has joined #ceph
[14:20] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:26] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[14:34] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[14:44] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[14:50] * rongze_ (~rongze@117.79.232.204) has joined #ceph
[14:52] * Sysadmin88 (~IceChat77@94.1.37.151) Quit (Quit: Some folks are wise, and some otherwise.)
[14:55] * rongze (~rongze@211.155.113.166) Quit (Ping timeout: 480 seconds)
[15:02] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[15:06] * Shmouel (~Sam@fny94-12-83-157-27-95.fbx.proxad.net) has joined #ceph
[15:08] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) has joined #ceph
[15:09] <mozg> hello guys
[15:09] <LCF> hi
[15:09] <mozg> I am planning to upgrade 0.67.4 to the latest stable
[15:09] <mozg> was wondering if there is anything that I should watch out for?
[15:10] <LCF> base on docu looks easy
[15:10] <LCF> but I will wait little more with new stable release
[15:10] <mozg> yeah?
[15:10] <mozg> is there a particular reason for that?
[15:10] <mozg> i thought that the current stable has been out for about 3 weeks already
[15:11] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:15] * rongze_ (~rongze@117.79.232.204) Quit (Remote host closed the connection)
[15:15] <LCF> mozg: no special reason
[15:16] <mozg> Is anyone using Emperor?
[15:16] <mozg> was wondering how it works comparing to the 0.67 release?
[15:16] <LCF> mozg: don't have problems now and no special features for me in 0.72
[15:16] <mozg> LCF, are you on 0.67?
[15:16] * rongze (~rongze@117.79.232.204) has joined #ceph
[15:17] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[15:18] * danieagle (~Daniel@186.214.73.228) Quit (Quit: inte+ e Obrigado Por tudo mesmo! :-D)
[15:22] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit ()
[15:22] * xdeller_ (~xdeller@95-31-29-125.broadband.corbina.ru) Quit (Quit: Leaving)
[15:24] <LCF> mozg: 0.67.4
[15:26] * rongze_ (~rongze@211.155.113.166) has joined #ceph
[15:26] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[15:27] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:32] * rongze__ (~rongze@211.155.113.166) has joined #ceph
[15:32] <mozg> LCF, same here
[15:33] * rongze (~rongze@117.79.232.204) Quit (Read error: Connection reset by peer)
[15:34] * rongze (~rongze@117.79.232.204) has joined #ceph
[15:34] * rongze_ (~rongze@211.155.113.166) Quit (Read error: Connection reset by peer)
[15:34] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[15:40] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[15:40] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[15:40] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[15:40] * rongze__ (~rongze@211.155.113.166) Quit (Read error: Operation timed out)
[15:41] * sleinen1 (~Adium@2001:620:0:26:4cd8:fe0a:f6c2:d3af) has joined #ceph
[15:43] * rongze_ (~rongze@211.155.113.166) has joined #ceph
[15:43] * rongze__ (~rongze@211.155.113.166) has joined #ceph
[15:44] * madkiss (~madkiss@089144194208.atnat0003.highway.a1.net) has joined #ceph
[15:44] * rongze (~rongze@117.79.232.204) Quit (Read error: Connection reset by peer)
[15:48] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[15:49] * rongze_ (~rongze@211.155.113.166) Quit (Read error: Operation timed out)
[15:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:55] * yanzheng (~zhyan@134.134.139.74) Quit (Ping timeout: 480 seconds)
[16:01] * BillK (~BillK-OFT@58-7-73-113.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[16:03] * BillK (~BillK-OFT@106-68-121-93.dyn.iinet.net.au) has joined #ceph
[16:05] * rongze__ (~rongze@211.155.113.166) Quit (Remote host closed the connection)
[16:06] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[16:06] * fred_ (~fred@c83-248-221-150.bredband.comhem.se) has joined #ceph
[16:07] * rongze (~rongze@211.155.113.166) has joined #ceph
[16:10] <fred_> Been searching for the answer to a question i have regarding striping. Say i want to write 1GB of data to a cluster. Not using striping shouldn't the pg's and replicas distribute the writes mor or less evenly. Also when reading the same data. Doesn't ceph read from different pg's and by that get high throuput? Does ceph read from "slave" pg replicas or only the primary pg replica? If so why do we need striping at all?
[16:11] * glzhao (~glzhao@118.195.65.67) Quit (Quit: leaving)
[16:15] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[16:17] * mnash (~chatzilla@66-194-114-178.static.twtelecom.net) has joined #ceph
[16:17] * nwat (~textual@eduroam-255-104.ucsc.edu) has joined #ceph
[16:18] * sleinen1 (~Adium@2001:620:0:26:4cd8:fe0a:f6c2:d3af) Quit (Quit: Leaving.)
[16:18] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[16:22] * dxd828 (~dxd828@94.116.110.8) has joined #ceph
[16:27] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[16:27] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) Quit (Quit: Ex-Chat)
[16:28] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) Quit (Ping timeout: 480 seconds)
[16:29] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[16:34] * lightspeed (~lightspee@82-68-190-217.dsl.in-addr.zen.co.uk) has joined #ceph
[16:41] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[16:45] * Sysadmin88 (~IceChat77@94.1.37.151) has joined #ceph
[16:47] * mschiff (~mschiff@port-2150.pppoe.wtnet.de) has joined #ceph
[16:49] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[16:49] * rongze_ (~rongze@211.155.113.166) has joined #ceph
[16:55] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[16:56] * rongze (~rongze@211.155.113.166) Quit (Read error: Operation timed out)
[17:01] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[17:03] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[17:07] * mtanski_ (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[17:07] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Read error: Connection reset by peer)
[17:07] * mtanski_ is now known as mtanski
[17:11] * dxd828 (~dxd828@94.116.110.8) Quit (Quit: Computer has gone to sleep.)
[17:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[17:15] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[17:20] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[17:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[17:31] * rongze_ (~rongze@211.155.113.166) Quit (Remote host closed the connection)
[18:02] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[18:02] * ChanServ sets mode +o joao|lap
[18:02] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) has joined #ceph
[18:04] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[18:07] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[18:08] * Sysadmin88 (~IceChat77@94.1.37.151) Quit (Quit: Never put off till tomorrow, what you can do the day after tomorrow)
[18:08] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit ()
[18:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:25] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:26] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) has joined #ceph
[18:28] * ron-slc2 (~ron-slc@173-165-129-125-utah.hfc.comcastbusiness.net) has joined #ceph
[18:30] <ron-slc2> Has anybody else seen the Dumpling ceph-mds crash, when a Cuttlefish ceph-fuse client connects? I just determined, every time i power on a client with Cuttlefish client, it's the cause of crash.
[18:32] * rongze (~rongze@117.79.232.204) has joined #ceph
[18:32] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[18:35] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[18:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:44] * rongze (~rongze@117.79.232.204) Quit (Ping timeout: 480 seconds)
[18:45] * topro_ (~tobi@ip-109-43-3-234.web.vodafone.de) has joined #ceph
[18:46] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Ping timeout: 480 seconds)
[18:46] <pmatulis> ron-slc2: i'm not sure if heterogeneous versions is supported. can you upgrade the client to see if the version plays a role in the problem?
[18:48] <ron-slc2> Yes, I do need to upgrade the client. But the problem seems that I must also Force unmount all the ~10 or so clients, bring the mds back up, then restart all clients again. it seems a connection from a Dumpling client is poison and all clients must be reset...
[18:49] <ron-slc2> And the problem is, there is always another old client which i forget, and sure enough it gets powered on and takes the mds down.
[18:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:51] <pmatulis> did this start happening when you upgraded the cluster (C -> D) or was it always happening?
[18:51] <ron-slc2> This only began upon upgrade from C -> D
[18:51] <ron-slc2> upgrade of the ceph-mds
[18:52] <ron-slc2> it seems the client-reconnect is a little funky.
[18:52] <ron-slc2> because even after I umount -f on the client side, I see a reconnect by client.15739 10.99.10.50:0/6212 after 0.017596 in the mds log
[18:53] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[18:53] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Quit: Leaving)
[18:53] <pmatulis> i don't use cephfs exactly b/c of all the problems i hear about. it seems the only thing to do is to make sure you upgrade all the clients then power on MDS
[18:53] <pmatulis> most of the problems, mind you, are upgrade problems
[18:54] <ron-slc2> lol I hear ya man. But when you have a boat load of them, you always forget one.
[18:55] <ron-slc2> thought I was done fighting this 2 weeks ago, but lo-and behold a VM fires up, and *!!!*
[18:55] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[18:55] <pmatulis> ah
[18:56] * pmatulis makes a note of the importance of keeping track of clients
[18:57] <pmatulis> maybe firewall them unless they have been upgraded, but you need to know what to firewall
[18:57] <ron-slc2> yea..... would be *MUCH* nicer, if ceph-mds would have a "disconnect() if version < ##" method.
[18:58] <ron-slc2> it's a 1-liner fix.
[18:58] <pmatulis> consider opening a RFE bug
[18:59] <ron-slc2> yea. need to. Just trying to find what finally makes this thing come up..
[19:00] <pmatulis> you might want to try capturing some logs when it breaks
[19:06] <ron-slc2> and it seems, I have to force reboot any client using kernel-cephfs client.
[19:06] <ron-slc2> lest, its reconnects seem to also crash Dumpling MDS
[19:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[19:12] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) Quit (Quit: Leaving...)
[19:14] <ron-slc2> yep, confirmed. when the "situation" happens. restarting Dumpling ceph-mds fails during the rejoin and client-reconnect phase. Forcing me to SSH into every client, and kill -s 9 the ceph-fuse; *OR* if kernel cephfs client power off. after this mds recovers properly, and clients can join -new
[19:15] * perfectsine (~perfectsi@if01-gn01.dal05.softlayer.com) Quit (Ping timeout: 480 seconds)
[19:16] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[19:19] * nwat (~textual@eduroam-255-104.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
[19:25] * dpippenger (~riven@cpe-76-166-208-83.socal.res.rr.com) Quit (Quit: Leaving.)
[19:26] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[19:26] * Cube (~Cube@netblock-72-25-110-178.dslextreme.com) has joined #ceph
[19:29] * alexxy[home] (~alexxy@2001:470:1f14:106::2) has joined #ceph
[19:29] * alexxy (~alexxy@2001:470:1f14:106::2) Quit (Read error: Connection reset by peer)
[19:30] * Cube (~Cube@netblock-72-25-110-178.dslextreme.com) Quit ()
[19:35] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[19:36] * rongze (~rongze@117.79.232.236) has joined #ceph
[19:37] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) Quit (Quit: Computer has gone to sleep.)
[19:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[19:43] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) has joined #ceph
[19:48] * rongze (~rongze@117.79.232.236) Quit (Ping timeout: 480 seconds)
[19:54] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:03] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[20:06] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) Quit (Quit: themgt)
[20:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:19] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:20] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[20:26] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
[20:28] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) Quit (Quit: Leaving...)
[20:30] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) has joined #ceph
[20:31] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) Quit ()
[20:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:54] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[20:57] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:58] * madkiss (~madkiss@089144194208.atnat0003.highway.a1.net) Quit (Quit: Leaving.)
[21:11] * topro_ (~tobi@ip-109-43-3-234.web.vodafone.de) Quit (Quit: Konversation terminated!)
[21:12] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[21:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[21:13] * perfectsine (~perfectsi@if01-gn01.dal05.softlayer.com) has joined #ceph
[21:15] * ponyofdeath (~vladi@cpe-75-80-165-117.san.res.rr.com) Quit (Remote host closed the connection)
[21:19] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[21:23] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:29] * diegows (~diegows@190.190.11.42) has joined #ceph
[21:34] * sleinen (~Adium@2001:620:0:26:e849:18a3:6e91:c5b4) has joined #ceph
[21:41] * rongze (~rongze@211.155.113.166) has joined #ceph
[21:42] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[21:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[21:48] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:50] * rongze (~rongze@211.155.113.166) Quit (Ping timeout: 480 seconds)
[21:59] <fred_> anyone know if higher replication factor improves read performance?
[22:02] * DarkAce-Z (~BillyMays@50.107.53.200) has joined #ceph
[22:06] * DarkAceZ (~BillyMays@50.107.53.200) Quit (Ping timeout: 480 seconds)
[22:07] <Gugge-47527> fred_: it does not
[22:07] <Gugge-47527> fred_: the random nature of the system makes you read from all osd's, no matter what your replication factor is
[22:08] <Gugge-47527> a higher replication factor gives you more write io, and then you actually has less io left to give you read performance :)
[22:09] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[22:10] <fred_> Gugge-47527: So if i have 3 level of repl factor set and store only a single object.. accessing that object from lots and lots of clients will only put load on a single server?
[22:12] <fred_> Is this due to consistency requirements/locking or something? I mean .. the clients have access to the crushmap and should also have knowledge of where the replicas are?
[22:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:20] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) has joined #ceph
[22:20] * ChanServ sets mode +o scuttlemonkey
[22:28] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:29] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[22:36] <mrjack_> fred_: whats the point in reading also replicas?
[22:37] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) Quit (Quit: Computer has gone to sleep.)
[22:38] <fred_> mrjack_: Higher IO, perhaps lower latency.
[22:39] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[22:40] <fred_> mrjack_: If the primary pg is in a remove DC. Why should the client not read from the replica in the local DC?
[22:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:47] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[22:48] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) has joined #ceph
[22:53] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:53] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[22:55] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[23:01] * DarkAce-Z is now known as DarkAceZ
[23:12] * yanzheng (~zhyan@jfdmzpr02-ext.jf.intel.com) has joined #ceph
[23:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:13] <Kioob> fred_: maybe you can use stripping
[23:17] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[23:17] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit ()
[23:19] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:20] <fred_> Kioob: That would still waste bandwidth and increase latency since some stripes could reside in some other DC. For databases we use one primary and several replicas at different locations closer to the end user. All writes goes to the primary. Reads will depending on client setting go local or to the primary depending on the consistency requirement. Using ceph a client uses a secondary if the primary goes down right? Shouldn't you be able to
[23:20] <fred_> add a setting that "prefer" replicas closer to the client?
[23:21] <Gugge-47527> As it is right now, only the primary replica is read
[23:21] <Gugge-47527> i guess because you always know the primary has a complete copy :)
[23:21] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) Quit (Quit: Leaving...)
[23:24] <fred_> Would the osd on the replica support a "read" call from the client if the client was modified or would the osd need modification as well?
[23:25] * sarob (~sarob@2601:9:7080:13a:24ca:f9c7:96e3:c813) has joined #ceph
[23:27] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) has joined #ceph
[23:28] * xmltok (~xmltok@cpe-23-240-222-226.socal.res.rr.com) Quit ()
[23:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:43] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[23:44] * dxd828 (~dxd828@host86-172-58-128.range86-172.btcentralplus.com) has joined #ceph
[23:45] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit (Ping timeout: 480 seconds)
[23:59] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.