#ceph IRC Log

Index

IRC Log for 2013-11-21

Timestamps are in GMT/BST.

[0:02] <gregsfortytwo> new release every three months ;)
[0:03] * mschiff (~mschiff@85.182.236.82) Quit (Remote host closed the connection)
[0:03] <Nats_> i applaud your commitment to doing point releases
[0:03] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[0:03] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:03] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[0:03] * mxmln (~mxmln@195.222.244.63) Quit (Quit: mxmln)
[0:09] <joshd> L2SHO: not yet http://tracker.ceph.com/issues/190
[0:09] * indeed (~indeed@206.124.126.33) has joined #ceph
[0:11] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[0:15] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has joined #ceph
[0:15] * rongze (~rongze@117.79.232.205) has joined #ceph
[0:16] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has left #ceph
[0:17] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) Quit (Quit: Leaving)
[0:19] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) has joined #ceph
[0:19] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) Quit ()
[0:19] * tarfik (~AndChat26@public-gprs587412.centertel.pl) has joined #ceph
[0:20] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:20] <tarfik> Hi,
[0:20] <tarfik> I plan to delete 2 buckets, 5M and 15M files.
[0:20] <tarfik> This can be dangerous if I do it via:
[0:20] <tarfik> radosgw-admin --bucket=largebucket1 --purge-objects bucket rm
[0:20] <tarfik> ?
[0:23] * rongze (~rongze@117.79.232.205) Quit (Ping timeout: 480 seconds)
[0:27] * tarfik (~AndChat26@public-gprs587412.centertel.pl) Quit (Quit: Bye)
[0:30] <JoeGruher> does rbd stand for something specific? rados block device?
[0:32] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:33] <gregsfortytwo> exactly
[0:34] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Quit: ...)
[0:35] <JoeGruher> :)
[0:35] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[0:38] <joao|lap> Pauline, please file a ticket with that? Add all information you may think would be useful, ideally even how to reproduce it, and I'll take a look in the morning :)
[0:38] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Ping timeout: 480 seconds)
[0:38] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[0:40] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit ()
[0:40] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[0:41] * julian (~julianwa@125.70.134.54) Quit (Read error: Connection reset by peer)
[0:41] * julian (~julianwa@125.70.134.54) has joined #ceph
[0:42] <kl4m> Hi, I'm trying to create a ceph cluster for the first time and I can't get it to clean health, it's stuck in inactive,unclean. It's the same problem as another on the list: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-November/005692.html
[0:44] <kl4m> Here's my ceph status http://pastebin.com/raw.php?i=qpAxhSYX basically the same thing
[0:44] <aarontc> kl4m: how many OSDs do you have?
[0:44] <kl4m> The problem is, I can't figure what the cluster is expecting to fix this
[0:44] <bandrus1> kl4m: http://eu.ceph.com/docs/wip-6090/start/quick-ceph-deploy/#create-a-cluster
[0:45] <aarontc> kl4m: I can't remember the details, but for some reason I'm thinking you can't get very far with < 3 OSDs?
[0:45] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:45] <bandrus1> your OSDs exist on the same node, therefore you have to adjust CRUSH algorithm to place replicas on the same node
[0:46] <bandrus1> by default, they will be placed on separate nodes for obvious redundancy reasons
[0:47] <kl4m> forget the "glusterfs" node names, it's just trolling :)
[0:48] * KindTwo (KindOne@h214.36.186.173.dynamic.ip.windstream.net) has joined #ceph
[0:48] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[0:48] * KindTwo is now known as KindOne
[0:49] <kl4m> The same node? I have 1 osd per machine, on 2 machines. Do you mean the same nodes as monitors?
[0:50] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[0:51] * dxd828 (~dxd828@host-92-24-127-29.ppp.as43234.net) Quit (Quit: Computer has gone to sleep.)
[0:51] * indeed (~indeed@206.124.126.33) Quit (Quit: Leaving...)
[0:55] <aarontc> kl4m: I think he thought you had all your OSDs on the same physical host
[0:55] <aarontc> kl4m: are you using the default CRUSH map?
[0:55] <kl4m> aarontc: yes
[0:56] <aarontc> can you tail the logs from both OSDs and pastebin them?
[0:57] <Psi-Jack> Whoah wait a sec..
[0:57] <bandrus1> kl4m, aarontc: yes, I assumed both OSDs on the same node, as that is the most common cause of the issue you are describing.
[0:57] <Psi-Jack> Ceph 0.72 includes tgt support directly?
[0:57] * Cube (~Cube@66-87-66-150.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[1:00] <bandrus1> kl4m: what is the size of your pools?
[1:00] * yanzheng (~zhyan@134.134.137.73) Quit (Remote host closed the connection)
[1:01] <kl4m> bandrus1: it's size 2, min_size 1 at the moment but size 1 didnt change the situation earlier
[1:01] <Psi-Jack> heh. Looks like I'll be planning an upgrade to 0.72 soon.
[1:02] <aarontc> Psi-Jack: where did you see that? I'm curious now
[1:02] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[1:02] <Psi-Jack> aarontc: Release.
[1:02] <Psi-Jack> http://ceph.com/docs/master/release-notes/
[1:02] * Cube (~Cube@66-87-66-51.pools.spcsdns.net) has joined #ceph
[1:03] <Psi-Jack> tgt: improved support for iSCSI via upstream tgt
[1:03] <aarontc> I thought that still required iSCSI services external to ceph, they have just improved rbd's support for being the backing devices?
[1:03] * mikedawson_ (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[1:04] <kl4m> aarontc, bandrus1: Here's the logs on an OSD on startup http://pastebin.com/raw.php?i=mBCdYmjk
[1:04] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[1:04] * mikedawson_ is now known as mikedawson
[1:04] * shang (~ShangWu@119-46-166-36.static.asianet.co.th) has joined #ceph
[1:05] <aarontc> kl4m: can you dump your crush map and pastebin the decoded output from crushtool?
[1:06] <kl4m> aarontc: http://pastebin.com/raw.php?i=kg7MUjFi
[1:07] <aarontc> kl4m: There you go :) Neither of your OSDs are in the CRUSH map
[1:07] <kl4m> I tried to put devices manually in that crushmap also, no change
[1:08] <aarontc> kl4m: you need a 'ceph osd crush create-or-move <OSD-ID> <size> host=<hostname>' for each of your OSDs
[1:10] <aarontc> (I'm not 100% sure that's ALL you need, but that should get the OSDs into your CRUSH map and with the proper weights)
[1:10] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:11] <kl4m> What does that size refer to?
[1:11] <aarontc> technically, it's the OSD weight but the convention is to use the OSD size in TiB as the weight
[1:11] <aarontc> so if you have 4TB drives, the size is 3.67
[1:12] <aarontc> sorry, 3.64
[1:12] <Psi-Jack> Heh, I... Dunno.. I still have issues shutting down due to cephfs mounts.
[1:12] <dmick> Psi-Jack: no, Ceph doesn't change for tgt
[1:13] <Psi-Jack> I see.
[1:13] <dmick> I just updated the tgt bs_rbd backend to be nicer
[1:13] <Psi-Jack> Ahhh
[1:13] <dmick> it just got announced in the same cycle as Ceph
[1:15] <Psi-Jack> Yeah. I have one server that uses cephfs, and evertime I shut down, it stalls umounting. This seems to be an annoyingly common issue on several of my vm guests.
[1:15] <Psi-Jack> More often than not, I have to force the vm to power off to get it to shut down.
[1:16] <aarontc> Psi-Jack: I don't think that's related to iSCSI. you may have an issue like I did, where the init system is stopping the network interfaces before unmounting the filesystem
[1:19] <kl4m> aarontc: # ceph osd crush create-or-move 0 3.64 root=default host=glusterfs4
[1:19] <kl4m> aarontc: create-or-move updating item name 'osd.0' weight 3.64 at location {host=glusterfs4,root=default} to crush map # same for the other node, not it's HEALTH_OK
[1:19] <kl4m> *now
[1:19] <aarontc> kl4m: cool
[1:19] <kl4m> any idead how I could set this as a default?
[1:20] <aarontc> kl4m: if you use the Ceph init script to start the OSD, it calls the create-or-move by default
[1:20] <Pauline> joao|lap: will do
[1:20] <kl4m> ah, the ceph-osd-all never seems to start anything, I need to check what it's trying to do
[1:21] <kl4m> (it's a setup with the chef cookbook btw)
[1:21] <aarontc> I've never used chef
[1:26] * dosaboy (~dosaboy@65.93.189.91.lcy-01.canonistack.canonical.com) Quit (Read error: Operation timed out)
[1:26] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[1:27] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[1:31] <kl4m> looks like I need to mount the osd data at /var/lib/ceph/osd/[cluster name]-[osd id] and the upstart scripts are happy
[1:32] <aarontc> kl4m: sounds right. I don't use upstart myself
[1:32] * shang (~ShangWu@119-46-166-36.static.asianet.co.th) Quit (Remote host closed the connection)
[1:32] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[1:32] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[1:32] <dmick> kl4m: there are magic udev things to make that happen if you have GPT labels and the right partition types
[1:32] <dmick> (as ceph-deploy sets up)
[1:33] <aarontc> dmick: When will that magic support LVM? :)
[1:33] <kl4m> That's the problem for me unfortunately, I can't use a device such as sdb directly
[1:34] <kl4m> a static xfs mount will be fine
[1:34] <aarontc> kl4m: Yeah, I use LVM (behind encrypted LUKS device), so I have all my OSDs in the config file and let the init script mount everything for me
[1:34] <Psi-Jack> aarontc: heh.. yep. Seems you are correct, CentOS 6.4 is disconnecting ethX before _netdev mounts
[1:36] * clayb (~kvirc@proxy-nj2.bloomberg.com) Quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/)
[1:36] <aarontc> Psi-Jack: Yeah, OpenRC on Gentoo does that too, I had to hack i a bit to fix the problem
[1:37] <Psi-Jack> aarontc: Thanks for that. Will investigate further. But yeah, makes sense. :)
[1:39] <kl4m> alright, everything survives a reboot and comes back clean, time to go home now. Thanks for the help
[1:40] <aarontc> no problem, kl4m
[1:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:53] * yanzheng (~zhyan@jfdmzpr04-ext.jf.intel.com) has joined #ceph
[1:54] <dmick> aarontc: beats me, I can't even spell LVM :)
[1:55] <aarontc> dmick: lol :)
[1:56] <aarontc> my understanding of udev is pretty fuzzy or I'd try it myself... right now my biggest cluster has 22 OSDs so it's no big deal to edit a config, but I anticipate that growing fast next year
[2:01] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[2:02] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[2:05] <Nats_> can anyone offer insight into what debug ms = 0/0 does?
[2:08] * kl4m (~kl4m@66.254.36.166) Quit (Quit: Leaving...)
[2:09] <dmick> for the "ms", or messenger, subsystem, the console and memory log levels are set to 0
[2:09] <dmick> meaning that a message must be of level 0 to be shown in the log or kept in memory for dumping on crash
[2:09] <dmick> so basically almost off
[2:10] <Nats_> is the messenger subsystem particularly chatty? i find it makes a fair difference
[2:10] <dmick> what makes a difference for what?
[2:11] <dmick> and, it can be
[2:11] * mwarwick (~mwarwick@2407:7800:200:1011:3e97:eff:fe91:d9bf) has joined #ceph
[2:11] <Nats_> i find adding "debug ms = 0/0" to my [client] section makes about a 20% difference to iops
[2:11] <dmick> that seems larger than I'd expect
[2:11] <aarontc> what are the default debug ms values?
[2:12] <Nats_> my setup is all ssd so perhaps thats why
[2:12] <dmick> one can observe them with ceph daemon <daemon> config show
[2:12] <dmick> or examining src/common/config_opts.cc
[2:12] <dmick> er, .h
[2:13] <dmick> 0/5, looks like
[2:13] <Nats_> i found a list of all them on a mailing list set to 0/0 , by commenting them out i found the 'debug ms' is the only one that makes a significant difference
[2:13] <aarontc> cool, I might have to play with that
[2:14] <aarontc> my biggest performance issue is writes starving reads, but I haven't deep-dived anything since adding instrumentation
[2:14] * yanzheng (~zhyan@jfdmzpr04-ext.jf.intel.com) Quit (Remote host closed the connection)
[2:15] <dmick> You can set to 5/5 to see how much is spewed
[2:16] <dmick> but at 0/5 it's only going into a memory buffer, so I'm surprised, like I say
[2:16] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:17] * xarses (~andreww@64-79-127-122.static.wiline.com) Quit (Ping timeout: 480 seconds)
[2:18] <Nats_> thanks
[2:21] * scuttlemonkey (~scuttlemo@c-174-51-178-5.hsd1.co.comcast.net) has joined #ceph
[2:21] * ChanServ sets mode +o scuttlemonkey
[2:22] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:24] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[2:27] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[2:31] * yanzheng (~zhyan@jfdmzpr03-ext.jf.intel.com) has joined #ceph
[2:33] * scuttlemonkey (~scuttlemo@c-174-51-178-5.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[2:34] * aliguori (~anthony@74.202.210.82) Quit (Remote host closed the connection)
[2:37] * sarob (~sarob@ip-64-134-239-22.public.wayport.net) has joined #ceph
[2:37] * haomaiwang (~haomaiwan@117.79.232.238) Quit (Ping timeout: 480 seconds)
[2:37] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[2:39] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[2:41] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:46] <Psi-Jack> Hmm
[2:46] <Psi-Jack> So, apparently centos's netfs doesn't give a rats arse what has _netdev. heh
[2:47] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[2:49] * sjusthm (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Remote host closed the connection)
[2:53] * haomaiwang (~haomaiwan@211.155.113.245) has joined #ceph
[2:54] <aarontc> Psi-Jack: I ... moved to other distros a long time ago. I have been really frustrated with RHEL-based distributions in the past :)
[2:54] <Psi-Jack> Heh
[2:55] <Psi-Jack> Eh, I figured it out, i just did a little hacking to the netfs init script, and it looks... Good. I just added ceph support specifically.
[2:55] * angdraug (~angdraug@64-79-127-122.static.wiline.com) Quit (Quit: Leaving)
[2:57] <Psi-Jack> Basically CentOS uses _netdev as block devices, specifically. not network filesystems.
[2:58] <Psi-Jack> LOL
[2:58] <Psi-Jack> Funny. rsyslog was using my cephfs mount, and that stops /after/ netfs. great..
[2:58] * yy-nm (~Thunderbi@122.224.154.38) Quit (Quit: yy-nm)
[2:58] * sarob (~sarob@ip-64-134-239-22.public.wayport.net) Quit (Remote host closed the connection)
[2:59] <aarontc> lol, that's pretty awesome
[2:59] <Psi-Jack> Yeaaah... LOl
[2:59] <aarontc> is _netdev a standard GNU mount feature, or a RH "addition"?
[2:59] <Psi-Jack> Well, my VM i'm working on just happends to be my central log server. Where all my syslogs go.
[3:00] <Psi-Jack> aarontc: Hmmm, standard, i've known about it since before RH.
[3:00] <aarontc> Psi-Jack: Interesting, I've never encountered it before today :)
[3:01] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[3:02] <Psi-Jack> heh
[3:02] <Psi-Jack> Either way, if this netfs mostly fixes everything else, besides this one host, I'm just going to use salt to keep netfs "modified" accross the board. ;)
[3:03] * dpippenger (~riven@cpe-76-166-208-83.socal.res.rr.com) Quit (Remote host closed the connection)
[3:03] * nhm (~nhm@c-174-51-178-5.hsd1.co.comcast.net) has joined #ceph
[3:03] * ChanServ sets mode +o nhm
[3:03] <Psi-Jack> On this one server, obviously I need to adjust the stop for rsyslog to be before netfs, or such.
[3:03] <aarontc> Psi-Jack: That's why I run Gentoo.. when I have to change core stuff, I don't want to have to manually replicate that over hundreds of machines :)
[3:03] <Psi-Jack> aarontc: salt. Easy. :)
[3:04] <Psi-Jack> Nothing to do with distro. :)
[3:05] <aarontc> Hm, guess I'm not familiar with it. "SaltStack"?
[3:05] <Psi-Jack> Correct.
[3:05] <Psi-Jack> It's like puppet, cfengine, etc.. But better.
[3:07] <aarontc> I'll have to look at it. My entire infrastructure could use an overhaul, I've looked at things like puppet and they have always seemed like either huge overkill, or had too many -isms I didn't want to follow
[3:07] <Psi-Jack> heh
[3:07] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:08] <Psi-Jack> salt, is pretty simple. You can use it for configuration management and templating, and you can use it for running commands to multiple servers in parallel.
[3:08] <aarontc> I currently use "ndo" for that, it's dead simple. ~/.ndo/<hostlist>, then "ndo <hostlist> cmd"
[3:09] <aarontc> like "ndo ceph-all wget -O /etc/ceph/ceph.conf http://ceph-master/ceph.conf"
[3:11] <aarontc> since I use Gentoo, all my configuration is codified in terms of Portage packages in my site overlay, so it's as easy as "ndo all emerge --update <package-name>" and done :)
[3:12] <aarontc> But, I like new toys so I'm gonna play with SaltStack
[3:13] <Psi-Jack> nice.
[3:13] <Psi-Jack> Override for LSB init in /etc/chkconfig.d/rsyslog to depend on $remote_fs ;)
[3:13] <Psi-Jack> And.. That worked.
[3:14] * jefferai (~quassel@corkblock.jefferai.org) has joined #ceph
[3:14] <aarontc> Cool :)
[3:14] <aarontc> now if only it was that easy to teach Ubuntu :/
[3:14] <Psi-Jack> Yeah, chkconfig is more powerful than people tend to realize. :0
[3:15] <Psi-Jack> hah! Ubuntu? Upstart. nuff said.
[3:15] <Psi-Jack> ubuntu didn't know LSB so they made upstart. ;)
[3:15] <aarontc> Yeah, exactly, I haven't the time or desire to mess with upstart
[3:15] <aarontc> I chuckle a little every time I run "/etc/init.d/<service> restart" and it whines "you know you can say 'service <name> restart, right?!'"
[3:16] <Psi-Jack> But yeah. checked and confirmed, my manual netfs hack and chkconfig.d override did the trick for this server. ;)
[3:16] <Psi-Jack> I know upstart, myself. I use it on CentOS sometimes. ;)
[3:16] <Psi-Jack> In fact, I wrote upstart init scripts for ceph daemons for CentOS. :)
[3:17] <aarontc> Nice. I'm planning to decommission all the Ubuntu servers in the next 8 to 10 months so I don't see any point in starting now
[3:17] <Psi-Jack> Mainly because, if ceph-osd or something stops, because of oom killer or something, upstart will respawn it up to a certain threshold to keep it running.
[3:17] <Psi-Jack> openrc still doesn't have watchdogging.
[3:18] * sarob (~sarob@207.239.83.246) has joined #ceph
[3:18] <aarontc> true, but to date I haven't needed that. if something goes down, Zabbix alerts me, and I can investigate why. I guess restarting things wouldn't hurt in most cases, but most things are set up redundantly so it's not a time-critical problem
[3:19] <aarontc> I found, years ago, that running out of disk space or RAM seemed to be the problem 99% of the time, and Zabbix alerts about that proactively, so I rarely experience a spurious daemon crash
[3:23] * rongze (~rongze@117.79.232.220) has joined #ceph
[3:26] <Psi-Jack> There. netfs patched and deployed. ;)
[3:27] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[3:30] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[3:30] * shang (~ShangWu@42.64.11.40) has joined #ceph
[3:34] * glzhao (~glzhao@118.195.65.67) has joined #ceph
[3:45] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[3:46] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:50] * nhm (~nhm@c-174-51-178-5.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[3:55] * sarob (~sarob@207.239.83.246) Quit (Remote host closed the connection)
[3:56] * sarob (~sarob@207.239.83.246) has joined #ceph
[4:01] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[4:01] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[4:04] * sarob (~sarob@207.239.83.246) Quit (Ping timeout: 480 seconds)
[4:05] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[4:11] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[4:11] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has left #ceph
[4:11] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:16] * yanzheng (~zhyan@jfdmzpr03-ext.jf.intel.com) Quit (Remote host closed the connection)
[4:20] * otisspud (~otisspud@198.15.79.50) has joined #ceph
[4:28] * tsnider (~tsnider@198.95.226.40) has joined #ceph
[4:29] * julian (~julianwa@125.70.134.54) Quit (Quit: afk)
[4:29] * julian (~julianwa@125.70.134.54) has joined #ceph
[4:31] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[4:34] * DarkAce-Z (~BillyMays@50.107.53.200) has joined #ceph
[4:34] * shang (~ShangWu@42.64.11.40) Quit (Read error: Connection reset by peer)
[4:38] * yanzheng (~zhyan@jfdmzpr04-ext.jf.intel.com) has joined #ceph
[4:39] * DarkAceZ (~BillyMays@50.107.53.200) Quit (Ping timeout: 480 seconds)
[4:44] * shang (~ShangWu@42.64.11.40) has joined #ceph
[4:45] * cephn00b (~cephn00b@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[4:48] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:51] * BillK (~BillK-OFT@58-7-117-112.dyn.iinet.net.au) Quit (Read error: Connection reset by peer)
[4:56] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[5:00] * BillK (~BillK-OFT@220-253-134-36.dyn.iinet.net.au) has joined #ceph
[5:00] * shang (~ShangWu@42.64.11.40) Quit (Read error: Connection reset by peer)
[5:02] * cephn00b (~cephn00b@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: cephn00b)
[5:02] * shang (~ShangWu@42.64.11.40) has joined #ceph
[5:03] * fireD (~fireD@93-142-241-115.adsl.net.t-com.hr) Quit (Remote host closed the connection)
[5:04] <tsnider> gatherkeys seemed to work. when I activate OSDs on the 2nd storage node I get "librados: client.bootstrap-osd authentication error (1) Operation not permitted
[5:04] <tsnider> [swift14][ERROR ] Error connecting to cluster: PermissionError" . resolution / debug ideas appreciated. the install is scripted and was working prior to a cluster move and IP address change. I can access storage nodes from the monitor node on the private network by name without a problem and have done ssh-key-copy from the monitor (controll/install) node to the storage nodes.
[5:05] * fireD (~fireD@93-142-209-202.adsl.net.t-com.hr) has joined #ceph
[5:13] * Hakisho_ (~Hakisho@p4FC2647A.dip0.t-ipconnect.de) has joined #ceph
[5:13] * Hakisho (~Hakisho@0001be3c.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:13] * Hakisho_ is now known as Hakisho
[5:14] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[5:27] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[5:28] * sarob (~sarob@ip-64-134-239-22.public.wayport.net) has joined #ceph
[5:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[5:34] * tsnider (~tsnider@198.95.226.40) Quit (Quit: Leaving.)
[5:36] * sarob (~sarob@ip-64-134-239-22.public.wayport.net) Quit (Ping timeout: 480 seconds)
[5:39] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[5:47] * BillK (~BillK-OFT@220-253-134-36.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[5:48] * BillK (~BillK-OFT@106-68-105-6.dyn.iinet.net.au) has joined #ceph
[5:49] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[5:56] * shang (~ShangWu@42.64.11.40) Quit (Read error: Connection reset by peer)
[6:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[6:04] * BillK (~BillK-OFT@106-68-105-6.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[6:05] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[6:06] * BillK (~BillK-OFT@106-68-45-41.dyn.iinet.net.au) has joined #ceph
[6:19] * BillK (~BillK-OFT@106-68-45-41.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[6:20] * BillK (~BillK-OFT@124-148-83-35.dyn.iinet.net.au) has joined #ceph
[6:21] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[6:27] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[6:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[6:38] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[6:38] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[6:42] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[6:46] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[6:47] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[6:51] * scuttlemonkey (~scuttlemo@2601:1:bf00:dda:5497:7e93:ac79:f10b) has joined #ceph
[6:51] * ChanServ sets mode +o scuttlemonkey
[6:55] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[6:58] * julian_ (~julian@125.70.134.54) has joined #ceph
[7:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[7:02] * julian (~julianwa@125.70.134.54) Quit (Quit: afk)
[7:06] * joao|lap (~JL@a79-168-11-205.cpe.netcabo.pt) Quit (Ping timeout: 480 seconds)
[7:09] * bloodice (blinker@50-89-11-53.res.bhn.net) Quit (Quit: Whats this button do? FUCK!)
[7:10] * AfC (~andrew@2407:7800:200:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[7:11] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[7:25] * BillK (~BillK-OFT@124-148-83-35.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[7:26] * BillK (~BillK-OFT@124-169-202-187.dyn.iinet.net.au) has joined #ceph
[7:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[7:34] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[7:36] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[7:38] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[7:39] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[7:40] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit ()
[7:45] * scuttlemonkey (~scuttlemo@2601:1:bf00:dda:5497:7e93:ac79:f10b) Quit (Ping timeout: 480 seconds)
[7:48] * KindTwo (KindOne@h108.37.28.71.dynamic.ip.windstream.net) has joined #ceph
[7:50] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[7:50] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[7:50] * KindTwo is now known as KindOne
[7:54] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Ping timeout: 480 seconds)
[7:56] * AfC (~andrew@2407:7800:200:1011:2ad2:44ff:fe08:a4c) has joined #ceph
[7:59] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[8:12] * AfC (~andrew@2407:7800:200:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[8:17] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:22] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[8:25] * mwarwick (~mwarwick@2407:7800:200:1011:3e97:eff:fe91:d9bf) Quit (Ping timeout: 480 seconds)
[8:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[8:33] * mwarwick (~mwarwick@2407:7800:200:1011:6e88:14ff:fe48:57e4) has joined #ceph
[8:38] * Sysadmin88 (~IceChat77@94.1.37.151) Quit (Quit: Copywight 2007 Elmer Fudd. All wights wesewved.)
[8:42] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:50] * xdeller (~xdeller@95-31-29-125.broadband.corbina.ru) has joined #ceph
[8:51] * julian_ (~julian@125.70.134.54) Quit (Remote host closed the connection)
[8:54] * julian (~julian@125.70.134.54) has joined #ceph
[8:55] * JoeGruher (~JoeGruher@134.134.139.72) Quit (Remote host closed the connection)
[8:55] * tziOm (~bjornar@194.19.106.242) Quit (Read error: Connection reset by peer)
[8:55] * rendar (~s@host215-179-dynamic.23-79-r.retail.telecomitalia.it) has joined #ceph
[8:57] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[9:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[9:00] * mattt_ (~textual@94.236.7.190) has joined #ceph
[9:08] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[9:14] * mschiff (~mschiff@tmo-107-13.customers.d1-online.com) has joined #ceph
[9:19] * shang (~ShangWu@42-64-11-40.dynamic-ip.hinet.net) has joined #ceph
[9:23] * xdeller (~xdeller@95-31-29-125.broadband.corbina.ru) Quit (Ping timeout: 480 seconds)
[9:26] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[9:29] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[9:29] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[9:30] * mxmln (~mxmln@212.79.49.65) has joined #ceph
[9:30] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[9:32] * xdeller (~xdeller@95-31-29-125.broadband.corbina.ru) has joined #ceph
[9:32] * shang (~ShangWu@42-64-11-40.dynamic-ip.hinet.net) Quit (Read error: Connection reset by peer)
[9:33] * mxmln_ (~mxmln@212.79.49.65) has joined #ceph
[9:36] * shang (~ShangWu@42.64.11.40) has joined #ceph
[9:38] * mxmln (~mxmln@212.79.49.65) Quit (Ping timeout: 480 seconds)
[9:38] * mxmln_ is now known as mxmln
[9:41] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[9:42] * mwarwick (~mwarwick@2407:7800:200:1011:6e88:14ff:fe48:57e4) Quit (Quit: Leaving.)
[9:47] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[9:47] * ChanServ sets mode +v andreask
[9:47] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[9:50] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[9:53] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Read error: Operation timed out)
[9:55] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[9:59] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[10:01] * yanzheng (~zhyan@jfdmzpr04-ext.jf.intel.com) Quit (Remote host closed the connection)
[10:01] * shang (~ShangWu@42.64.11.40) Quit (Read error: Connection reset by peer)
[10:03] * ScOut3R (~ScOut3R@catv-89-133-22-210.catv.broadband.hu) has joined #ceph
[10:04] * xdeller (~xdeller@95-31-29-125.broadband.corbina.ru) Quit (Quit: Leaving)
[10:09] * thomnico (~thomnico@2a01:e35:8b41:120:1d0a:e211:da12:d8c1) has joined #ceph
[10:12] * mwarwick (~mwarwick@110-174-133-236.static.tpgi.com.au) has joined #ceph
[10:12] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[10:17] * dosaboy (~dosaboy@host81-152-8-140.range81-152.btcentralplus.com) has joined #ceph
[10:19] * dosaboy (~dosaboy@host81-152-8-140.range81-152.btcentralplus.com) Quit ()
[10:19] * dosaboy (~dosaboy@host81-152-8-140.range81-152.btcentralplus.com) has joined #ceph
[10:21] * glzhao (~glzhao@118.195.65.67) Quit (Quit: leaving)
[10:29] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[10:35] * Cube (~Cube@66-87-66-51.pools.spcsdns.net) Quit (Quit: Leaving.)
[10:35] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) has joined #ceph
[10:38] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[10:43] * dosaboy (~dosaboy@host81-152-8-140.range81-152.btcentralplus.com) Quit (Quit: leaving)
[10:43] * dosaboy (~dosaboy@65.93.189.91.lcy-01.canonistack.canonical.com) has joined #ceph
[10:45] * thomnico (~thomnico@2a01:e35:8b41:120:1d0a:e211:da12:d8c1) Quit (Quit: Ex-Chat)
[10:49] * Cube (~Cube@66-87-66-51.pools.spcsdns.net) has joined #ceph
[10:53] * mschiff_ (~mschiff@tmo-107-13.customers.d1-online.com) has joined #ceph
[10:53] * mschiff (~mschiff@tmo-107-13.customers.d1-online.com) Quit (Ping timeout: 480 seconds)
[11:00] * Cube (~Cube@66-87-66-51.pools.spcsdns.net) Quit (Quit: Leaving.)
[11:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:06] * xdeller (~xdeller@91.218.144.129) has joined #ceph
[11:07] * pressureman (~pressurem@62.217.45.26) has joined #ceph
[11:13] * WarrenUsui (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) Quit (Read error: Connection reset by peer)
[11:13] * WarrenUsui (~Warren@2607:f298:a:607:a8d9:5660:cbb1:e25d) has joined #ceph
[11:16] * KindTwo (KindOne@h48.0.40.162.dynamic.ip.windstream.net) has joined #ceph
[11:18] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[11:18] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[11:18] * KindTwo is now known as KindOne
[11:24] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[11:26] * shang (~ShangWu@118.175.161.38) has joined #ceph
[11:26] * mattch (~mattch@pcw3047.see.ed.ac.uk) has joined #ceph
[11:27] * mschiff_ (~mschiff@tmo-107-13.customers.d1-online.com) Quit (Ping timeout: 480 seconds)
[11:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:39] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) Quit (Read error: Connection reset by peer)
[11:42] * mwarwick (~mwarwick@110-174-133-236.static.tpgi.com.au) Quit (Quit: Leaving.)
[11:43] * ksingh (~Adium@teeri.csc.fi) has joined #ceph
[11:43] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[11:43] <ksingh> are there any geeks on duty today ??
[11:43] <ksingh> or any one from community who can help me with this error
[11:43] <ksingh> [root@rdo yum.repos.d]# qemu-img create -f rbd rbd:data/foo 10G
[11:43] <ksingh> Unknown file format 'rbd'
[11:43] <ksingh> [root@rdo yum.repos.d]#
[11:44] <ksingh> i am using centos 6.4
[11:44] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[11:46] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[11:49] <ksingh> any one online
[11:49] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[11:52] * yanzheng (~zhyan@134.134.137.75) has joined #ceph
[11:53] * rongze (~rongze@117.79.232.220) Quit (Remote host closed the connection)
[11:53] <Gugge-47527> ksingh: sounds like your qemu-img does not know the rbd type
[11:53] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[11:53] * rongze (~rongze@117.79.232.220) has joined #ceph
[11:53] <ksingh> yes
[11:53] <ksingh> any idea how to fix this
[11:54] <ksingh> i am following ceph documentation but still no luck with this problem
[11:55] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[11:57] * shang (~ShangWu@118.175.161.38) Quit (Read error: Operation timed out)
[11:57] <mattt> ksingh: which version of qemu?
[11:58] <mattt> ksingh: http://wiki.qemu.org/ChangeLog/0.14#ceph.2Frbd
[11:59] <ksingh> qemu-img version 0.12.1
[11:59] <mattt> well there's your prob
[12:00] <ksingh> does this version does not have support to RBD
[12:00] <ksingh> ?
[12:01] * rongze (~rongze@117.79.232.220) Quit (Ping timeout: 480 seconds)
[12:01] <mattt> ksingh: i think so, the wiki says support was added in 0.14
[12:02] <ksingh> Matt : then how can i get the newest version of Qemu
[12:02] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[12:02] <ksingh> do you have rpm link for centos
[12:04] <mattt> ksingh: not sure :-/
[12:05] <ksingh> i cannot find it on ceph rpm repository
[12:05] <mattt> ksingh: can you not use fedora or ubuntu or something that has a supportive qemu ?
[12:05] <ksingh> No thats not the correct way
[12:05] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[12:06] <mattt> ksingh: then what's the alternative? :)
[12:07] * ScOut3R (~ScOut3R@catv-89-133-22-210.catv.broadband.hu) Quit (Read error: Operation timed out)
[12:09] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:09] * allsystemsarego (~allsystem@5-12-240-115.residential.rdsnet.ro) has joined #ceph
[12:14] <fireD> http://ceph.com/packages/qemu-kvm/centos/x86_64/ however I'm not sure if those are the latest they have
[12:23] * ScOut3R (~ScOut3R@212.96.46.212) has joined #ceph
[12:27] <andreask> is there a debug mode for ceph-deploy?
[12:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[12:33] * i_m (~ivan.miro@deibp9eh1--blueice1n1.emea.ibm.com) has joined #ceph
[12:42] * mschiff (~mschiff@tmo-108-218.customers.d1-online.com) has joined #ceph
[12:46] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:57] <ksingh> hello andreask
[12:57] <ksingh> i am stucked in no mans land , help me
[12:57] <ksingh> i am using this document http://ceph.com/docs/master/install/install-vm-cloud/
[12:58] * sleinen (~Adium@2001:620:0:26:c956:2e00:403f:d5cf) has joined #ceph
[12:58] <ksingh> after installation i am geting error
[12:58] <ksingh> Unknown file format 'rbd'
[12:58] <ksingh> the version is qemu-img version 0.12.1
[12:59] <ksingh> i want to use RBD with openstack , do i necessarily need qemu on openstack node ??
[12:59] <ksingh> please suggest me what should i do
[13:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:10] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) has joined #ceph
[13:12] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:13] * shang (~ShangWu@42.64.11.40) has joined #ceph
[13:20] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) Quit (Quit: Ex-Chat)
[13:20] * tsnider (~tsnider@ip68-102-128-87.ks.ok.cox.net) has joined #ceph
[13:23] * tsnider1 (~tsnider@198.95.226.40) has joined #ceph
[13:25] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) has joined #ceph
[13:25] * shang (~ShangWu@42.64.11.40) Quit (Read error: Connection reset by peer)
[13:27] * mschiff (~mschiff@tmo-108-218.customers.d1-online.com) Quit (Ping timeout: 480 seconds)
[13:28] * tsnider (~tsnider@ip68-102-128-87.ks.ok.cox.net) Quit (Ping timeout: 480 seconds)
[13:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:36] * ShaunR (~ShaunR@staff.ndchost.com) has joined #ceph
[13:36] * ShaunR- (~ShaunR@staff.ndchost.com) Quit (Read error: Connection reset by peer)
[13:38] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:39] <andreask> ksingh: looks like you are using a way to old qemu version
[13:39] <andreask> ksingh: and yes, you need an up2date version to use it with openstack
[13:41] * mozg (~andrei@host217-46-236-49.in-addr.btopenworld.com) Quit (Quit: Ex-Chat)
[13:42] * wogri_risc (~wogri_ris@ro.risc.uni-linz.ac.at) has joined #ceph
[13:44] * BillK (~BillK-OFT@124-169-202-187.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[13:47] * tsnider1 (~tsnider@198.95.226.40) Quit (Quit: Leaving.)
[13:52] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[13:53] * rongze (~rongze@117.79.232.237) has joined #ceph
[13:54] <ksingh> andreask : where to get updated version , i searched every where on web
[13:54] <ksingh> but no luck
[13:54] * ScOut3R_ (~ScOut3R@212.96.46.212) has joined #ceph
[13:54] <ksingh> can you help me with the correct version
[13:54] <andreask> ksingh: whats your os
[13:54] * ScOut3R_ (~ScOut3R@212.96.46.212) Quit (Remote host closed the connection)
[13:54] * ScOut3R_ (~ScOut3R@212.96.46.212) has joined #ceph
[13:55] * ScOut3R_ (~ScOut3R@212.96.46.212) Quit (Remote host closed the connection)
[13:55] * ScOut3R_ (~ScOut3R@212.96.46.212) has joined #ceph
[13:58] <ksingh> centos 6.4
[13:59] * ScOut3R (~ScOut3R@212.96.46.212) Quit (Ping timeout: 480 seconds)
[13:59] * dvanders (~dvanders@dvanders-air.cern.ch) Quit (Read error: Operation timed out)
[14:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:01] * rongze (~rongze@117.79.232.237) Quit (Ping timeout: 480 seconds)
[14:03] * glzhao (~glzhao@118.195.65.67) has joined #ceph
[14:04] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[14:04] * rongze (~rongze@117.79.232.205) has joined #ceph
[14:05] * alfredodeza (~alfredode@c-24-131-46-23.hsd1.ga.comcast.net) has joined #ceph
[14:09] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[14:11] <loicd> alfredodeza: the more I used ceph-deploy the more I understand why you say it would be extraordinarily difficult to support all possible use cases.
[14:11] <alfredodeza> loicd: we would need to clone myself a few dozen times
[14:11] <alfredodeza> :)
[14:11] <loicd> norris alfredodozen
[14:11] <kraken> Some magicans can walk on water, alfredodozen can swim through land.
[14:12] <loicd> alfredodeza: that works it seems ;-)
[14:12] <alfredodeza> kraken haiku add_use sevens extraordinarily
[14:12] <kraken> make your linter cry
[14:12] <kraken> extraordinarily
[14:12] <kraken> this CDS stuff
[14:12] <loicd> Oo
[14:12] <loicd> kraken haiku ceph-deploy
[14:13] <loicd> kraken haiku ceph-deploy is not a haiku
[14:13] <alfredodeza> lol
[14:13] <loicd> hum
[14:13] <loicd> kraken haiku usage
[14:13] <alfredodeza> loicd: try: kraken help haiku
[14:13] <loicd> kraken help haiku
[14:13] * kraken whispers to loicd
[14:14] <loicd> kraken haiku blam loicd
[14:14] <loicd> kraken haiku blame loicd
[14:14] <alfredodeza> I don't think I have blamed anyone (yet)
[14:14] <alfredodeza> :)
[14:14] <loicd> kraken haiku blame add five loicd
[14:15] <loicd> kraken haiku blame five loicd
[14:15] <loicd> kraken haiku blame sevens loicd
[14:15] <loicd> kraken haiku add_use sevens loicd
[14:15] <kraken> look at the juju
[14:15] <kraken> loicd
[14:15] <kraken> teach kraken to turn
[14:16] <alfredodeza> that is not a seven!
[14:16] * xdeller (~xdeller@91.218.144.129) Quit (Ping timeout: 480 seconds)
[14:17] <alfredodeza> kraken: haiku remove sevens loicd
[14:17] <kraken> right-o
[14:17] <alfredodeza> o/
[14:17] <kraken> \o
[14:17] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[14:17] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:18] <loicd> :-)
[14:19] <andreask> ksing: you can try the one from http://ceph.com/packages/ceph-extras/rpm/rhel6/x86_64/ ... the kvm packages with the "async" ... but I have not tested them with dumpling or emporer ...
[14:19] * michalefty (~oftc-webi@proxy1.t-online.net) has joined #ceph
[14:19] <andreask> ksingh: ^^^^
[14:20] <alfredodeza> loicd: I created a ceph-deploy blueprint for the upcoming CDS: http://wiki.ceph.com/01Planning/02Blueprints/Firefly/ceph-deploy
[14:20] <ksingh> Thanks andreask , you are very helpful
[14:20] <andreask> ksingh: you are welcome
[14:20] <alfredodeza> it has a couple of items there, you might be interested to give it a quick read
[14:20] <alfredodeza> nothing fancy really
[14:21] <andreask> alfredodeza: is ceph-deploy also tested with RHEL ?
[14:21] * xdeller (~xdeller@91.218.144.129) has joined #ceph
[14:21] <alfredodeza> I don't think we have RHEL boxes
[14:21] <alfredodeza> so I would say no
[14:21] <andreask> looks like, yes ;-)
[14:21] <alfredodeza> what do you mean
[14:21] * michalefty (~oftc-webi@proxy1.t-online.net) Quit (Remote host closed the connection)
[14:22] <andreask> just tried it on a Centos 6.4 cluster
[14:22] <alfredodeza> iirc the last time we discussed this was that we create and destroy a *lot* of VMs for testing, it would be difficult to have licenses for RHEL repos for ephemeral machines
[14:22] <alfredodeza> andreask: ah we do test on CentOS
[14:22] <alfredodeza> :)
[14:27] * tsnider (~tsnider@nat-216-240-30-23.netapp.com) has joined #ceph
[14:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:39] * fouxm (~fouxm@2a04:2500:0:b00:ad13:35ae:440e:4ae1) has joined #ceph
[14:42] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:44] * loicd looking
[14:44] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[14:46] <loicd> alfredodeza: definitely interested :-D
[14:46] <loicd> fouxm: hi
[14:46] <fouxm> hi loicd !!
[14:48] <loicd> fouxm: benchmarking is discussed in the context of http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Ceph-Brag and http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Test_Automation mostly at the moment.
[14:50] <loicd> mikedawson ( not here atm ) is currently working on benchmarks and will be most interested to discuss with you, I think
[14:51] <loicd> I would very much like some kind of method to run benchmarks because it will be needed for erasure code
[14:52] <loicd> apeters is working on erasure code benchmarks and wrote a standalone binary. But it's unclear in which context it should be run, how results should be collected etc.etc.
[14:52] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[14:55] * glzhao (~glzhao@118.195.65.67) Quit (Quit: leaving)
[14:55] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[14:56] <fouxm> ok, thanks for the update loicd, I'am having a look at the links you posted and l'll try to catch mike as soon as i see him online
[14:58] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[14:59] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[14:59] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[15:00] * andreask (~andreask@zid-vpnn016.uibk.ac.at) has joined #ceph
[15:00] * ChanServ sets mode +v andreask
[15:01] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:01] * japuzzo (~japuzzo@129.42.208.174) has joined #ceph
[15:03] <loicd> fouxm: test_automation does not say much so far, but it's a topic we've discussed repeatedly during the weekly meetings.
[15:03] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[15:05] * andreask (~andreask@zid-vpnn016.uibk.ac.at) Quit (Read error: Connection reset by peer)
[15:05] * xdeller (~xdeller@91.218.144.129) Quit (Quit: Leaving)
[15:06] * wogri_risc (~wogri_ris@ro.risc.uni-linz.ac.at) Quit (Remote host closed the connection)
[15:08] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:12] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[15:12] * mrjack (mrjack@office.smart-weblications.net) has joined #ceph
[15:16] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[15:16] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit ()
[15:19] * mschiff (~mschiff@tmo-108-218.customers.d1-online.com) has joined #ceph
[15:25] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[15:25] * ChanServ sets mode +v andreask
[15:30] * markbby (~Adium@168.94.245.1) has joined #ceph
[15:30] * rongze_ (~rongze@117.79.232.205) has joined #ceph
[15:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:31] * rongze (~rongze@117.79.232.205) Quit (Ping timeout: 480 seconds)
[15:46] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:49] * yanzheng (~zhyan@134.134.137.75) Quit (Remote host closed the connection)
[15:50] * xdeller (~xdeller@91.218.144.129) has joined #ceph
[15:53] * greg (~gkoch@38.86.161.178) has joined #ceph
[15:53] * gkoch (~gkoch@38.86.161.178) Quit (Read error: Connection reset by peer)
[16:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[16:01] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[16:02] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) has joined #ceph
[16:02] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) has joined #ceph
[16:03] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) Quit (Ping timeout: 480 seconds)
[16:03] * lightspeed (~lightspee@fw-carp-wan.ext.lspeed.org) has joined #ceph
[16:08] * allsystemsarego (~allsystem@5-12-240-115.residential.rdsnet.ro) Quit (Quit: Leaving)
[16:08] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[16:12] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[16:15] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[16:16] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[16:19] * mtanski_ (~mtanski@69.193.178.202) has joined #ceph
[16:24] * mtanski (~mtanski@69.193.178.202) Quit (Ping timeout: 480 seconds)
[16:24] * mtanski_ is now known as mtanski
[16:26] * raso (~raso@deb-multimedia.org) Quit (Quit: WeeChat 0.3.8)
[16:27] * raso (~raso@deb-multimedia.org) has joined #ceph
[16:31] <loicd> alfredodeza: being able to work the crush map from the command line is cool : dachary.org/?p=2536
[16:31] <loicd> alfredodeza: that's the other reason why I'm happy today ;-)
[16:31] <loicd> and not wine, mind you
[16:34] <alfredodeza> :)
[16:34] * alfredodeza looks
[16:36] * rongze (~rongze@211.155.113.169) has joined #ceph
[16:37] * greg (~gkoch@38.86.161.178) Quit (Ping timeout: 480 seconds)
[16:41] * rongze_ (~rongze@117.79.232.205) Quit (Ping timeout: 480 seconds)
[16:58] * japuzzo (~japuzzo@129.42.208.174) Quit (Read error: Connection reset by peer)
[16:58] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) Quit (Quit: Ex-Chat)
[16:59] * japuzzo (~japuzzo@129.42.208.174) has joined #ceph
[17:00] * japuzzo (~japuzzo@129.42.208.174) Quit ()
[17:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[17:00] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[17:03] * ksingh (~Adium@teeri.csc.fi) Quit (Quit: Leaving.)
[17:05] * ksingh (~Adium@teeri.csc.fi) has joined #ceph
[17:05] * ksingh (~Adium@teeri.csc.fi) Quit ()
[17:07] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) has joined #ceph
[17:08] * gregmark (~Adium@68.87.42.115) has joined #ceph
[17:08] * wrencsok (~wrencsok@wsip-174-79-34-244.ph.ph.cox.net) has left #ceph
[17:13] * ksingh (~Adium@teeri.csc.fi) has joined #ceph
[17:13] * wrencsok (~wrencsok@wsip-174-79-34-244.ph.ph.cox.net) has joined #ceph
[17:14] * kl4m (~kl4m@66.254.36.166) has joined #ceph
[17:16] * L2SHO (~L2SHO@office-nat.choopa.net) Quit (Quit: Leaving)
[17:17] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[17:19] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[17:20] * DarkAce-Z is now known as DarkAceZ
[17:20] * rongze (~rongze@211.155.113.169) Quit (Read error: Operation timed out)
[17:21] * haomaiwang (~haomaiwan@211.155.113.245) Quit (Ping timeout: 480 seconds)
[17:21] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[17:21] * ChanServ sets mode +v andreask
[17:23] * rongze (~rongze@117.79.232.221) has joined #ceph
[17:24] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[17:27] * haomaiwang (~haomaiwan@211.155.113.217) has joined #ceph
[17:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[17:34] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[17:35] * ScOut3R_ (~ScOut3R@212.96.46.212) Quit (Ping timeout: 480 seconds)
[17:37] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) Quit (Quit: Leaving.)
[17:37] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[17:39] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) has joined #ceph
[17:41] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[17:41] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) Quit ()
[17:42] * thomnico (~thomnico@2a01:e35:8b41:120:3809:3914:efec:eea1) Quit (Quit: Ex-Chat)
[17:42] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[17:45] <tsnider> QOTD -- can / how does performance benchmarks of a kernel mounted rbd image compare to performacne of the same image within a VM context? I keep debating with myself thinking those 'should be comparable' and then maybe not.
[18:00] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:01] * mschiff (~mschiff@tmo-108-218.customers.d1-online.com) Quit (Remote host closed the connection)
[18:02] * Sysadmin88 (~IceChat77@94.1.37.151) has joined #ceph
[18:06] * mattt_ (~textual@94.236.7.190) Quit (Quit: Computer has gone to sleep.)
[18:08] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:12] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[18:13] * ksingh (~Adium@teeri.csc.fi) has left #ceph
[18:14] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) Quit (Quit: Leaving.)
[18:24] * angdraug (~angdraug@64-79-127-122.static.wiline.com) has joined #ceph
[18:26] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[18:26] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[18:26] * sleinen (~Adium@2001:620:0:26:c956:2e00:403f:d5cf) Quit (Quit: Leaving.)
[18:27] * sleinen (~Adium@130.59.94.196) has joined #ceph
[18:30] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:30] * markbby (~Adium@168.94.245.1) Quit (Remote host closed the connection)
[18:32] * pressureman (~pressurem@62.217.45.26) Quit (Quit: Ex-Chat)
[18:33] <loicd> what is the parameter to change so that osd are not marked down as quickly on a high latency network ? ( high means >= 10ms )
[18:33] <mikedawson> loicd: ceph osd set nodown ?
[18:34] <loicd> mikedawson: hi ! I still want them to be marked down, only not as quickly.
[18:34] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:35] * sleinen (~Adium@130.59.94.196) Quit (Ping timeout: 480 seconds)
[18:36] <mikedawson> loicd: maybe osd_heartbeat_interval
[18:36] <loicd> mikedawson: thanks :-)
[18:36] * loicd reading http://ceph.com/docs/master/rados/configuration/mon-osd-interaction/
[18:37] <loicd> mikedawson: fouxm is working on ceph benchmarks and I suggested that you might be interested to know what he does.
[18:37] * topro_ (~tobi@ip-109-43-0-233.web.vodafone.de) has joined #ceph
[18:38] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) Quit (Quit: themgt)
[18:38] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[18:38] <mikedawson> absolutely! love to share info with you fouxm. Thanks loicd!
[18:38] * topro_ (~tobi@ip-109-43-0-233.web.vodafone.de) Quit (Read error: Connection reset by peer)
[18:39] * topro_ (~tobi@ip-109-43-0-233.web.vodafone.de) has joined #ceph
[18:39] <loicd> heartbeat interval is large ( >5sec) there must be another reason why my osds go down so often
[18:40] <mikedawson> loicd: mine only get marked down under heavy spindle contention caused by backfilling or recovering. I always see Slow Request warnings first
[18:42] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) has joined #ceph
[18:43] * clayb (~kvirc@199.172.169.97) has joined #ceph
[18:43] <loicd> mikedawson: I'm in a different situation : light workload but osd spread accross 4 datacenters with high latency ( > 10ms ) and slow links ( < 100Mb/s )
[18:43] <gregsfortytwo1> tsnider: it varies depending on workload; in general kernel and qemu-mounted rbd should perform similarly, but the userspace one can do some things that the kernel one can't, since it has a client-local cache
[18:44] <gregsfortytwo1> loicd: have you checked how full the pipes are?
[18:44] * mxmln (~mxmln@212.79.49.65) Quit (Ping timeout: 480 seconds)
[18:44] * i_m (~ivan.miro@deibp9eh1--blueice1n1.emea.ibm.com) Quit (Quit: Leaving.)
[18:44] <gregsfortytwo1> eg, is there just enough traffic happening that the heartbeat messages are taking too long to get from the sender's tcp buffers into the receiver's?
[18:48] * mschiff (~mschiff@85.182.236.82) has joined #ceph
[18:48] <topro_> gregsfortytwo1: hi, interested in some cephfs emperor experience feedback?
[18:48] <gregsfortytwo1> haha, sure
[18:49] <topro_> don't be afraid, looks really good a.t.m. but I was lucky to nut update before 0.72.1 i assume ;)
[18:49] <topro_> s/nut/not/
[18:50] <gregsfortytwo1> heh, some people didn't run into trouble; it depends how you went about it
[18:51] <gregsfortytwo1> but yes, that was fortunate for you/unfortunate for us
[18:51] <topro_> since upgrade to emperor things run really smooth over here (i have a cluster exporting /home using cephfs to about 10 linux-3.12 kernel clients)
[18:51] * mjeanson (~mjeanson@00012705.user.oftc.net) Quit (Remote host closed the connection)
[18:51] * mjeanson (~mjeanson@bell.multivax.ca) has joined #ceph
[18:51] <gregsfortytwo1> that's great!
[18:51] <topro_> 3 nodes, 9 osds, 3 mons, one mds
[18:52] <gregsfortytwo1> if yan zheng ever appears in the channel again you should let him know; he probably did most of the bugfixes between dumpling and emperor
[18:52] <topro_> thanks for all the hard work so far. all the remaining troubles I still had with dumpling seems to have been addressed
[18:52] <dmsimard> Anyone familiar with rgw integration with keystone ?
[18:53] <gregsfortytwo1> topro_: what were you running into with dumpling?
[18:53] <topro_> anyway I'm experiencing a strange behaviour of three of my clients which are all connected using a single gbit link, I would like to discuss...
[18:55] <topro_> gregsfortytwo1: with dumpling, mainly _huge_ memory usage of daemons (OSD and MDS) which forced me to regularly restart services. and then restarting MDS was a nightmare because og cache initialisation bug. that was most noticable issues. and I'm not sure if with dumpling there still were data corruption issues (metadata in special)
[18:55] * JoeGruher (~JoeGruher@134.134.139.72) has joined #ceph
[18:55] <gregsfortytwo1> ah, got it
[18:56] <gregsfortytwo1> and I think those were all things that Yan fixed, yep
[18:56] <gregsfortytwo1> err, at least on the MDS side he did stuff about them
[18:57] * SvenPHX (~Adium@wsip-174-79-34-244.ph.ph.cox.net) Quit (Quit: Leaving.)
[18:57] * SvenPHX (~Adium@wsip-174-79-34-244.ph.ph.cox.net) has joined #ceph
[18:58] <topro_> for the strange behaviour i see right now, all 3 clients in one of our rooms (connected to cluster through a single 1 gbit link) simultaneously stall from time to time with processes hanging for about 30 secs in io-wait before resuming normal operation. all those three clients show some "mds went stale" in syslog at that same moment.
[18:58] <gregsfortytwo1> but not the others?
[18:58] <topro_> no other clients in other rooms ever showed such a thing
[18:58] <gregsfortytwo1> sounds like your network is either overloaded or you've got something going bad somehwere in the hardware ;)
[18:58] <topro_> I'm quite sure its a network issue, cabling to that room might be routed in parallel to power lines
[19:00] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[19:00] <loicd> gregsfortytwo1: no, I did not check how full the pipes. I'm not sure how to do that either :-) I thought the heartbeat had their own independant TCP connection for some reason.
[19:00] <gregsfortytwo1> loicd: they do, but it's obviously still going over the 100mbit connection
[19:00] <topro_> but anyway when monitoring the managed switch I don't see the link going down, nor am I loosing a single ping during that time. no other services, including voip-telephony over the same line, seem to suffer from that. so what I want to point at is that might the ceph protocol be very sensible to single package loss?
[19:00] <gregsfortytwo1> (sorry, I meant the internet pipe, not the messenger pipe)
[19:00] <gregsfortytwo1> topro_: shouldn't be, it's sitting on top of tcp
[19:01] <topro_> I do can see tcp-resend counters on the clients to grow during that stall
[19:01] <gregsfortytwo1> have you checked the cluster logs to see if the MDS is actually going stale, and the other clients just don't care?
[19:02] <gregsfortytwo1> but if you're seeing evidence of network issues I'd start there
[19:02] <gregsfortytwo1> keeping the connection alive should be easy enough even on a degraded link, but it's still a lot heavier-weight than just ping packets are
[19:03] <topro_> as I said I'm quite sure there are network issues. what I wanted to point out is to discuss if cephfs protocol reacts very sensible to such issues as the loss (if there is some) seems to be _very_ little but cephfs client will hang for about 30 seconds
[19:04] <gregsfortytwo1> topro_: I don't think the protocol should be; I'd need to see more information to draw any more conclusions
[19:05] * Sysadmin88 (~IceChat77@94.1.37.151) Quit (Quit: Few women admit their age. Few men act theirs.)
[19:05] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[19:05] <gregsfortytwo1> as a WAG, I bet the network is degrading more than you believe, and that's forcing the clients (and maybe the mds; check the cluster log) into a reconnect mode which is taking some time
[19:05] <topro_> btw. traffic on that link is far away from using the gbit link, so it is not a bandwidth issue. whats even more strange is that if I limit the link speed to 100mbit half-duplex that doesn't change a thing
[19:06] * ircolle (~Adium@2601:1:8380:2d9:e1b2:226e:c5e:41fd) has joined #ceph
[19:06] <topro_> any hint on how to see how much my link gets degraded during that time? bandwidth measurement?
[19:07] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[19:07] * sarob (~sarob@2601:9:7080:13a:6194:d042:a3f:b8a2) has joined #ceph
[19:08] <gregsfortytwo1> that's outside my expertise; sorry
[19:08] <topro_> never mind. of course thats off topic...
[19:08] <topro_> is there a predefined "timeout" which causes clients to reconnect?
[19:09] <topro_> or whats is the criterion for them to decide MDS is stale?
[19:10] <gregsfortytwo1> what's the exact message you're seeing?
[19:12] <gregsfortytwo1> topro_: and this is with kernel client, right?
[19:12] <topro_> from syslog: ceph: mds0 caps stale -> ceph: mds0 caps went stale, renewing -> ceph: mds0 caps stale -> ceph: mds0 caps renewed
[19:12] <topro_> whole process is taking about 15 seconds. but client already hangs for about 15 seconds before first syslog message appears
[19:13] <topro_> with linux-3.12 kernel client, right
[19:13] <topro_> as i previously reported fuse client is unusable performance-wise
[19:13] <gregsfortytwo1> oh, right
[19:14] <topro_> anyway, despite this network-induced issue, using kernel client, cephfs performs really great right now. so I don't want to complain in any way
[19:14] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[19:14] * ircolle (~Adium@2601:1:8380:2d9:e1b2:226e:c5e:41fd) Quit (Quit: Leaving.)
[19:14] <gregsfortytwo1> cool
[19:15] <gregsfortytwo1> bbiab, standup
[19:15] <topro_> np
[19:15] * sarob (~sarob@2601:9:7080:13a:6194:d042:a3f:b8a2) Quit (Ping timeout: 480 seconds)
[19:15] <topro_> thanks
[19:16] * sjusthm (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) has joined #ceph
[19:19] * xdeller (~xdeller@91.218.144.129) Quit (Quit: Leaving)
[19:20] * aliguori (~anthony@74.202.210.82) has joined #ceph
[19:23] * fouxm (~fouxm@2a04:2500:0:b00:ad13:35ae:440e:4ae1) Quit (Remote host closed the connection)
[19:25] * bandrus (~Adium@107.216.174.136) has joined #ceph
[19:30] <gregsfortytwo1> topro_: so there are two ways that message appears; either the client has taken more than the timeout period to request the MDS renew its caps, or the MDS has failed to respond in that amount of time
[19:30] <gregsfortytwo1> the timeout period is controlled by mds_session_timeout and defaults to 60s
[19:31] <gregsfortytwo1> you can change it by setting it on the monitors and waiting for a new mdsmap to be created (or doing something that creates one)
[19:31] * bandrus1 (~Adium@107.216.174.136) Quit (Ping timeout: 480 seconds)
[19:31] <gregsfortytwo1> but at 60 seconds there's got to be something going wrong on your machines or your network
[19:31] * bandrus1 (~Adium@107.216.174.136) has joined #ceph
[19:35] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) has joined #ceph
[19:36] * bandrus (~Adium@107.216.174.136) Quit (Ping timeout: 480 seconds)
[19:37] * sjm (~sjm@38.98.115.250) has joined #ceph
[19:40] * rongze (~rongze@117.79.232.221) Quit (Remote host closed the connection)
[19:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[19:44] <topro_> gregsfortytwo1: I was expecting you to report a value of ~100ms as that would have been a value I would have thought could happen. but 60 seconds are a really long time for the clients to be unable to talk to MDS. I know that even with a ping interval of 0.1 seconds I don't loos a single ping on that link... strange
[19:44] <gregsfortytwo1> yeah, I suspect there's a little more to it going on, dunno
[19:45] <gregsfortytwo1> I don't suppose they're VMs or that you've got wildly swinging clocks on them or something?
[19:45] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[19:47] * Cube (~Cube@12.248.40.138) has joined #ceph
[19:48] <topro_> plain hardware, no VMs involved. one of the clients had a missing ntp so time was some minutes off, but I corrected that some days ago so all clients have synced clocks now
[19:50] * aarontc (~aaron@static-50-126-79-226.hlbo.or.frontiernet.net) Quit (Ping timeout: 480 seconds)
[19:51] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[19:52] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[19:55] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[19:56] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[19:59] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:05] * sarob (~sarob@nat-dip29-wl-c.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:06] * dmsimard1 (~Adium@ap05.wireless.co.mtl.iweb.com) has joined #ceph
[20:08] * dmsimard2 (~Adium@108.163.152.66) has joined #ceph
[20:08] * sleinen (~Adium@2001:620:0:25:5803:4682:50de:4bd6) has joined #ceph
[20:11] * sleinen (~Adium@2001:620:0:25:5803:4682:50de:4bd6) Quit ()
[20:11] * rongze (~rongze@117.79.232.253) has joined #ceph
[20:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:12] * dmsimard (~Adium@108.163.152.2) Quit (Ping timeout: 480 seconds)
[20:13] * mtanski (~mtanski@69.193.178.202) Quit (Quit: mtanski)
[20:14] * dmsimard1 (~Adium@ap05.wireless.co.mtl.iweb.com) Quit (Read error: Operation timed out)
[20:20] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) Quit (Quit: themgt)
[20:20] * caleb_ (uid16207@id-16207.charlton.irccloud.com) has joined #ceph
[20:21] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) has joined #ceph
[20:22] * rongze (~rongze@117.79.232.253) Quit (Ping timeout: 480 seconds)
[20:23] * symmcom (~symmcom@184.70.203.22) has joined #ceph
[20:24] <symmcom> Can i simply use #ceph-deploy mon create <mon_id> to add new monitor into the cluster ?
[20:24] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:25] * The_Bishop (~bishop@2001:470:50b6:0:2d3e:12a2:8f13:d188) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[20:26] * nwat (~textual@eduroam-233-23.ucsc.edu) has joined #ceph
[20:28] <pmatulis> i don't think you can do <mon_id> if the monitor does not yet exist
[20:28] <pmatulis> and should to an 'install' beforehand too i think
[20:29] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[20:29] * xarses (~andreww@64-79-127-122.static.wiline.com) Quit (Ping timeout: 480 seconds)
[20:30] <symmcom> Yes, correct. #ceph-deploy install <mon-id> is first step. Then #ceph-deploy mon create. I am wondering if #ceph-deploy mon create is enough or do i need to go through old manual way such as #ceph mon -i -k -m etc etc
[20:31] <alfredodeza> symmcom: no need for that
[20:31] <alfredodeza> `ceph-deploy mon create {node}` should be sufficient
[20:34] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[20:37] * linuxkidd (~linuxkidd@ip72-193-217-254.lv.lv.cox.net) has joined #ceph
[20:38] * dmsimard2 (~Adium@108.163.152.66) Quit (Ping timeout: 480 seconds)
[20:39] <loicd> leseb: I'm collecting events for publishing them on http://ceph.com/community/events/. You did great during the OpenStack summit. When will you speak again ?
[20:40] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[20:40] <leseb> loicd: thank you :). I'll be doing almost the same presentation for the Openstack in action 4
[20:40] <loicd> leseb: URL ?
[20:41] * bandrus (~Adium@107.216.171.224) has joined #ceph
[20:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:42] <leseb> loicd: https://www.eventbrite.com/e/openstack-in-action-4-tickets-7645801799 please note that the schedule is going to be updated with my presentation
[20:43] * markbby (~Adium@168.94.245.4) has joined #ceph
[20:43] <loicd> leseb: great :-) Do you have an abstract ready by any chance ?
[20:44] <leseb> loicd: I do
[20:44] <leseb> loicd: where should I put it?
[20:44] <loicd> pastebinit ?
[20:44] <loicd> what's the time of your talk ?
[20:45] * bandrus1 (~Adium@107.216.174.136) Quit (Ping timeout: 480 seconds)
[20:45] * rongze (~rongze@117.79.232.221) has joined #ceph
[20:45] <leseb> loicd: http://pastebin.com/1rGAgTB7 , I think around 3pm
[20:46] <loicd> cool. Let me know when you're sure and I'll update the post.
[20:46] <leseb> loicd: I will :)
[20:47] <loicd> leseb: english or french ?
[20:48] <leseb> loicd: english
[20:49] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[20:50] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:51] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[20:51] * sarob (~sarob@nat-dip29-wl-c.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[20:52] * sarob (~sarob@nat-dip29-wl-c.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:52] <loicd> leseb: the title of your talk ?
[20:54] * rongze (~rongze@117.79.232.221) Quit (Ping timeout: 480 seconds)
[20:54] <loicd> I removed the last two paragraphs of http://pastebin.com/1rGAgTB7 as they seem to be meant for the reviewers of the submission rather than the actual abstract.
[20:55] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[20:56] <leseb> loicd: Ceph: de facto storage backend for OpenStack
[20:57] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[20:57] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[20:59] * sarob_ (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:59] <loicd> leseb: this is how it's going to look, let me know if you see a typo http://dachary.org/loic/ceph4/
[20:59] * sarob_ (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[20:59] * sarob_ (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[21:00] * sarob (~sarob@nat-dip29-wl-c.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[21:01] <loicd> if anyone around here knows of a ceph related event, please speak up ;-)
[21:03] * markbby (~Adium@168.94.245.4) Quit (Remote host closed the connection)
[21:05] * dxd828 (~dxd828@host-92-24-127-29.ppp.as43234.net) has joined #ceph
[21:06] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[21:06] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[21:11] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[21:11] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[21:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[21:13] <symmcom> one of my mon started giving this error whenever i try to run ceph command "Error initializing cluster client: Error". Any ideawhat caused it and what can i do? This is has been main MON for some time and nothing changed recently except i tried to run ceph-deploy mon create to create a new MON on a new machine.
[21:16] <dmsimard> leseb: I liked the presentation too, watched the video. I can't wait for presentations that are less about introductions and more about setting things up and what the integration really represents :)
[21:16] <topro_> do ceph daemons (MON/OSD/MDS) depend on kernel code and with that on kernel version in any way? i.e does it matter if I run ceph daemons on latest linux kernel?
[21:20] <loicd> dmsimard: :-)
[21:20] * bcat (~bcat@64-79-127-98.static.wiline.com) has joined #ceph
[21:20] <loicd> dmsimard: do you know of a Ceph speaker in the next following week ?
[21:21] <dmsimard> loicd: Me, at my job, but only my boss and colleagues are invited :P
[21:21] * kraken (~kraken@gw.sepia.ceph.com) Quit (Remote host closed the connection)
[21:21] <loicd> dmsimard: ahahah
[21:21] * kraken (~kraken@gw.sepia.ceph.com) has joined #ceph
[21:21] <bcat> hi, guys, I am confused about ceph's warning message, first it says HEALTH_WARN too few pgs per osd (13 < min 20);, then after I change the pg_num to higher number, it says pool data pg_num 128 > pgp_num 64;
[21:21] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[21:22] <bcat> so I tried to change the pg_num to 32 or 64, it won't do anything
[21:22] <bcat> any advice?
[21:26] * topro_ (~tobi@ip-109-43-0-233.web.vodafone.de) Quit (Quit: Konversation terminated!)
[21:28] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:30] <gregsfortytwo1> bcat: you've increase the number of PGs, now you need to increase the pgp_num so they move and actually balance out the data
[21:30] <gregsfortytwo1> same command, just pgp_num instead of pg_num
[21:30] <gregsfortytwo1> incurs some data movement
[21:31] <bcat> oh, I've missed a "p"
[21:31] <bcat> :) thanks for pointing it out
[21:32] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[21:33] <leseb> dmsimard: thanks, well probably during the next cephdays :)
[21:33] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[21:33] <leseb> loicd: reviewing
[21:34] <leseb> loicd: LGTM :) thanks for the effort
[21:36] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) has joined #ceph
[21:37] <loicd> leseb: cool
[21:37] <loicd> I'll wait for you signal to push "publish".
[21:37] <loicd> leseb: do you know of any other ceph related talks that would deserve to be announced on http://ceph.com/community/event/ ?
[21:37] * sleinen1 (~Adium@2001:620:0:25:b04d:e9c8:aaf4:7481) has joined #ceph
[21:38] * mxmln (~mxmln@212.79.49.66) has joined #ceph
[21:38] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[21:44] * sleinen (~Adium@77-58-245-10.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[21:47] <leseb> loicd: hum not really
[21:48] * gkoch (~gkoch@38.86.161.178) has joined #ceph
[21:49] * xarses_ (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[21:51] <gkoch> Hi all. I'm trying to run ceph-disk-prepare /dev/sdb and receiving ceph-disk: Error: getting cluster uuid from configuration failed. Did I overlook putting some kind of uuid in ceph.conf somewhere?
[21:52] <gkoch> first time ceph installer here
[21:52] * japuzzo (~japuzzo@pok2.bluebird.ibm.com) Quit (Quit: Leaving)
[21:52] * xarses (~andreww@64-79-127-122.static.wiline.com) Quit (Ping timeout: 480 seconds)
[21:54] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:57] <xarses_> gkoch: it is very much reccomended that you use ceph-deploy as a first time installer, it will take care of all the heavy lifting so you can play
[21:58] * xarses_ is now known as xarses
[21:58] <gkoch> I'm really looking to deploy this manually so we can deploy with puppet easier in production.
[21:59] <xarses> gkoch: understandable, there is still quite the learning curve, I'd still reccomend ceph-deploy, or use an existing puppet module to do the lifting so you don't end up with a un-usable deployment
[22:00] <alfredodeza> gkoch: ceph-deploy will tell you exactly what is doing at each point, so you will gain in know-how
[22:00] <alfredodeza> not only it does basic stuff but it will also help you understand common pitfalls that you might get into
[22:01] * kl4m_ (~kl4m@66.254.36.166) has joined #ceph
[22:03] <gkoch> I can give it a shot. I already have a mon running successfully. I assume I can proceede with ceph-deploy from there?
[22:05] <alfredodeza> I think you can
[22:09] * kl4m (~kl4m@66.254.36.166) Quit (Ping timeout: 480 seconds)
[22:10] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[22:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:13] <gkoch> Just tried to use ceph-deploy (after gathering all my keyrings). I actually got the same error message. Seems Imight have to blow away that whole config and start from scratch to get whatever it is I have missed in the config.
[22:14] <alfredodeza> gkoch: can you share some log output before you nuke everything?
[22:15] <gkoch> sure. Just ran ceph-deploy osd prepare ceph1:/var/lib/ceph/osd/osd.0 and it gave me ceph-disk: Error: getting cluster uuid from configuration failed with a traceback below that.
[22:15] <gkoch> Which logs would you like to see? osd.0.log is empty.
[22:15] * Underbyte (~jerrad@pat-global.macpractice.net) Quit (Ping timeout: 480 seconds)
[22:15] <alfredodeza> hrmn
[22:15] <alfredodeza> yeah what ceph-deploy gives back is usually enough
[22:16] <alfredodeza> gkoch: in your ceph.conf you don't have a `fsid` key?
[22:16] <alfredodeza> I think that is what your node is complaining about
[22:16] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) Quit (Quit: Leaving.)
[22:16] <alfredodeza> and yes, ceph-deploy would actually take care of that for you
[22:17] <gkoch> Yes, I don't have an fsid. Not really sure what that is at this point to be honest. The docs don't seem to really explain that - or maybe I just haven't found where it is explained.
[22:19] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[22:19] * ChanServ sets mode +v andreask
[22:19] * ircolle (~Adium@2601:1:8380:2d9:e1b2:226e:c5e:41fd) has joined #ceph
[22:20] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:20] <alfredodeza> gkoch: http://ceph.com/docs/master/rados/deployment/ceph-deploy-new/?highlight=fsid
[22:20] <alfredodeza> ah but that talks about ceph-deploy a bit
[22:21] <alfredodeza> I understand how that might be a bit confusing
[22:22] <gkoch> Yeah, seels like most pages end up telling you to just use ceph-deploy. We really gave up with ceph-deploy at step 1. It tries to download everything directly - where as we have synced the repos down to our local repo because the machines don't have internet access.
[22:22] * BillK (~BillK-OFT@124-169-202-187.dyn.iinet.net.au) has joined #ceph
[22:22] <gkoch> seems*
[22:23] <alfredodeza> ah correct, and funny you mention that, we just landed a feature to specify your own mirrors
[22:23] <alfredodeza> but alas, that is not released yet
[22:24] <alfredodeza> however
[22:24] <alfredodeza> there is a workaround for this, and that is, if you have your repo mirrors set on those machines you can tell ceph-deploy to not mangle those
[22:24] <alfredodeza> with: `ceph-deploy install --no-adjust-repos {node name}`
[22:29] <gkoch> interesting - maybe that would be the answer for me!
[22:32] * smiley (~smiley@205.153.36.170) has left #ceph
[22:34] * Underbyte (~jerrad@pat-global.macpractice.net) has joined #ceph
[22:34] * alram (~alram@cpe-76-167-50-51.socal.res.rr.com) has joined #ceph
[22:35] * mxmln_ (~mxmln@195.222.244.63) has joined #ceph
[22:39] * mxmln (~mxmln@212.79.49.66) Quit (Ping timeout: 480 seconds)
[22:39] * mxmln_ is now known as mxmln
[22:40] * kl4m_ (~kl4m@66.254.36.166) Quit (Remote host closed the connection)
[22:41] * kl4m (~kl4m@66.254.36.166) has joined #ceph
[22:41] <gkoch> Well, I added an fsid to the config and I think it got me a little further :-) getting a different error now.
[22:41] <gkoch> Could not create partition 2 from 2048001 to 2048000
[22:41] <gkoch> Error encountered; not saving changes.
[22:41] <gkoch> ceph-disk: Error: Command '['sgdisk', '--new=2:0:1000M', '--change-name=2:ceph journal', '--partition-guid=2:f7712bae-712a-4b1e-869b-06b60bc017d7', '--typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106', '--', '/dev/sdb']' returned non-zero exit status 4
[22:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:42] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has joined #ceph
[22:43] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[22:43] <Pedras> greetings
[22:43] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has left #ceph
[22:43] <Pedras> does anyone know if "preparing" disks with journals in another device is broken in ceph-deploy 1.3.2?
[22:44] <alfredodeza> Pedras: not that we are aware of
[22:44] <bcat> Pedras, I have no problem with it
[22:45] <bcat> what's your issue?
[22:45] <Pedras> so I am precreating the partitions for journals in the ssd
[22:45] <bcat> I have done 10+ installation with it in 2 days
[22:45] <Pedras> from my previous notes that seemed necessary
[22:46] <Pedras> then run something like ceph-deploy osd prepare HOST:sdd:sdc1
[22:46] <bcat> yes, you need to create partitions before disk preparing
[22:46] <Pedras> for example
[22:46] <bcat> incorrect order
[22:46] <bcat> ohoh
[22:46] <Pedras> that would explain it
[22:46] <bcat> never mind, go ahead, I thought it's ssd
[22:47] <bcat> it should be host:hd:sdd
[22:47] <Pedras> sdd is the hard disk
[22:47] <bcat> :) yeah, go ahead
[22:47] <Pedras> sdc1 is the first part of the ssd
[22:47] <Pedras> ahah .. I see your point
[22:47] <bcat> what's the issue specifically
[22:47] <Pedras> the journal partitions are being created with parted -a optimal /dev/$1 mklabel gpt
[22:47] * rongze (~rongze@117.79.232.176) has joined #ceph
[22:48] <bcat> are you sure you can use journal in gpt mode?
[22:48] <Pedras> the issue is that disk list after prepare/create doesn't show that sdc1 is the journal of sdd
[22:48] <Pedras> I am not sure but that was what was done before
[22:49] <Pedras> not emperor or ceph-deploy-1.3.2, but circa a couple months aho
[22:49] <Pedras> ago
[22:49] <bcat> it will show something like this "[ceph04][ERROR ] WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data"
[22:49] <Pedras> it does show that yes
[22:49] <Pedras> the ERROR followed by WARNING is odd
[22:50] <bcat> try this one
[22:50] <bcat> ll /var/lib/ceph/osd/ceph-0/
[22:50] <bcat> total 56
[22:50] <bcat> drwxr-xr-x 3 root root 198 Nov 21 10:43 ./
[22:50] <bcat> drwxr-xr-x 8 root root 4096 Nov 21 10:30 ../
[22:50] <bcat> -rw-r--r-- 1 root root 483 Nov 21 10:29 activate.monmap
[22:50] <bcat> -rw-r--r-- 1 root root 3 Nov 21 10:29 active
[22:50] <bcat> -rw-r--r-- 1 root root 37 Nov 21 10:28 ceph_fsid
[22:50] <bcat> drwxr-xr-x 23 root root 8192 Nov 21 12:43 current/
[22:50] <bcat> -rw-r--r-- 1 root root 37 Nov 21 10:28 fsid
[22:50] <bcat> lrwxrwxrwx 1 root root 9 Nov 21 10:28 journal -> /dev/sdg1
[22:50] <bcat> -rw------- 1 root root 56 Nov 21 10:29 keyring
[22:50] <bcat> -rw-r--r-- 1 root root 21 Nov 21 10:28 magic
[22:50] <bcat> -rw-r--r-- 1 root root 6 Nov 21 10:29 ready
[22:50] <bcat> -rw-r--r-- 1 root root 4 Nov 21 10:29 store_version
[22:50] <bcat> -rw-r--r-- 1 root root 42 Nov 21 10:29 superblock
[22:50] <bcat> -rw-r--r-- 1 root root 0 Nov 21 10:43 upstart
[22:50] <bcat> -rw-r--r-- 1 root root 2 Nov 21 10:29 whoami
[22:50] <bcat> see the journal? it's a symbolnk to ssd partition in my case
[22:50] <Pedras> yes indeed
[22:50] <Pedras> let me run again
[22:50] <Pedras> st d by
[22:51] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[22:53] <Pedras> purging all and such...
[22:53] * alfredodeza (~alfredode@c-24-131-46-23.hsd1.ga.comcast.net) has left #ceph
[22:55] * rongze (~rongze@117.79.232.176) Quit (Read error: Operation timed out)
[22:57] * Shmouel (~Sam@fny94-12-83-157-27-95.fbx.proxad.net) has joined #ceph
[23:00] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:04] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[23:05] * clayb (~kvirc@199.172.169.97) Quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/)
[23:08] * mozg (~andrei@host81-151-251-29.range81-151.btcentralplus.com) has joined #ceph
[23:10] <gkoch> Hi all - trying to activate an osd. disk mounts fine, journal is there as well. I receive a bunch of pipe().fault errors and then finally the following error messages:
[23:10] <gkoch> 2013-11-21 17:04:39.397092 7effe4adc700 0 monclient(hunting): authenticate timed out after 300
[23:10] <gkoch> 2013-11-21 17:04:39.397133 7effe4adc700 0 librados: osd.0 authentication error (110) Connection timed out
[23:10] <gkoch> Error connecting to cluster: Error
[23:10] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[23:11] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[23:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:12] * AfC (~andrew@2407:7800:200:1011:2ad2:44ff:fe08:a4c) has joined #ceph
[23:16] <andreask> gkoch: just a shot in the dark ... firewall?
[23:18] <gkoch> No, I am able to telnet to the mon on 6789
[23:18] <gkoch> I think my problem was that not all mon were running. Removed those that are not up right now from the config and now I'm just sitting here waiting for the osd to spit out something (either a success or failure).
[23:19] <andreask> your mons have quorum?
[23:22] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[23:23] * mxmln (~mxmln@195.222.244.63) Quit (Quit: mxmln)
[23:24] <gkoch> They might not... I think that's the issue.
[23:24] * bandrus1 (~Adium@c-98-238-148-252.hsd1.ca.comcast.net) has joined #ceph
[23:24] * sjm (~sjm@38.98.115.250) has left #ceph
[23:25] * nwat (~textual@eduroam-233-23.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[23:25] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:26] * gregsfortytwo1 (~Adium@cpe-172-250-69-138.socal.res.rr.com) has joined #ceph
[23:28] * bandrus (~Adium@107.216.171.224) Quit (Read error: Operation timed out)
[23:32] * alphe (~alphe@0001ac6f.user.oftc.net) has joined #ceph
[23:32] <alphe> hello folks
[23:32] <alphe> I don t know if someone can help me but here is my question :
[23:33] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[23:33] <alphe> I have a weird probleme with cephfs I start iton mon.0 and after a moment it jumps to another mon.2 ou mon.1 and that produce most of the data folder tree to disapear !
[23:34] <alphe> is there a way to fix that ?
[23:35] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Ping timeout: 480 seconds)
[23:38] * DarkAce-Z (~BillyMays@50.107.53.200) has joined #ceph
[23:39] * DarkAceZ (~BillyMays@50.107.53.200) Quit (Read error: Operation timed out)
[23:39] * rendar (~s@host215-179-dynamic.23-79-r.retail.telecomitalia.it) Quit ()
[23:40] * DarkAce-Z is now known as DarkAceZ
[23:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:45] * bandrus1 (~Adium@c-98-238-148-252.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[23:47] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Read error: Operation timed out)
[23:48] * CAPSLOCK2000 (~oftc@2001:610:748:1::8) Quit (Ping timeout: 480 seconds)
[23:48] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[23:48] * rongze (~rongze@117.79.232.253) has joined #ceph
[23:50] * dmsimard (~Adium@108.163.152.2) Quit (Read error: Operation timed out)
[23:50] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[23:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:53] * CAPSLOCK2000 (~oftc@2001:610:748:1::8) has joined #ceph
[23:57] * rongze (~rongze@117.79.232.253) Quit (Ping timeout: 480 seconds)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.