#ceph IRC Log

Index

IRC Log for 2013-11-18

Timestamps are in GMT/BST.

[0:04] * rongze (~rongze@117.79.232.205) Quit (Ping timeout: 480 seconds)
[0:04] <pmatulis> tsnider1: what OS is this again?
[0:04] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[0:05] <tsnider1> ubuntu 12.04
[0:06] <pmatulis> tsnider1: i can't find anything obvious in the strace but may want to do the same command with the working pool and try to find differences in the strace output
[0:06] <pmatulis> tsnider1: may also consider just turning of cephx for testing
[0:07] <pmatulis> tsnider1: vimdiff strace1 strace2, if you're a vim person
[0:08] <pmatulis> and do check your logs. also consider dialing up the log level (there is an rbd subsystem)
[0:08] <tsnider1> pmatulis: thx for the help. I've opened an inktank ticket also maybe they'll be able to tell me what's wrong
[0:09] <pmatulis> i don't think apparmor is involved. i use 'buntu as well and this semms very strange to me
[0:09] <tsnider1> I'll do the vimdiff thing .. familiar with it
[0:09] <pmatulis> great
[0:09] <tsnider1> thanks again
[0:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:17] * rendar (~s@95.238.179.113) Quit ()
[0:18] <pmatulis> tsnider1: http://tracker.ceph.com/issues/3285
[0:21] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:25] <jhujhiti> the output from ceph -w begins 'pgmap v[some incrementing number]'
[0:25] <jhujhiti> what happens when this number gets too large? does it wrap?
[0:28] * sprachgenerator (~sprachgen@c-50-141-192-36.hsd1.il.comcast.net) Quit (Quit: sprachgenerator)
[0:33] * jtang2 (~jtang@2001:468:1f07:c9:852b:b717:9276:9b85) has joined #ceph
[0:33] * jtang1 (~jtang@commodity-tech-212.sc13.org) Quit (Read error: Connection reset by peer)
[0:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[0:45] <aarontc> jhujhiti: I think they are working to prevent that from being an issue in the future
[0:46] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) has joined #ceph
[0:46] <jhujhiti> if it's a 64 bit number it's probably moot
[0:47] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[0:51] * yanzheng (~zhyan@jfdmzpr03-ext.jf.intel.com) Quit (Remote host closed the connection)
[0:53] <aarontc> jhujhiti: I asked about it a while back, since my cluster is already up to v2984717 and growing at a million a week, and the answer I got was basically don't worry about it :)
[0:55] * sjustlaptop (~sam@24-205-35-233.dhcp.gldl.ca.charter.com) Quit (Ping timeout: 480 seconds)
[0:57] * dxd828 (~dxd828@host-92-24-127-29.ppp.as43234.net) Quit (Quit: Computer has gone to sleep.)
[1:03] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[1:06] * jtang2 (~jtang@2001:468:1f07:c9:852b:b717:9276:9b85) Quit (Ping timeout: 480 seconds)
[1:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[1:15] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[1:19] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[1:20] * The_Bishop (~bishop@2001:470:50b6:0:f48c:56c0:e6f7:14c5) has joined #ceph
[1:23] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[1:24] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:32] * sarob (~sarob@2601:9:7080:13a:45aa:9d8c:e325:f160) has joined #ceph
[1:32] * sarob (~sarob@2601:9:7080:13a:45aa:9d8c:e325:f160) Quit (Remote host closed the connection)
[1:32] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[1:33] <AfC> This is definitely a stupid question, but I note that both radosgw and rbd seem to be enamored of stripping their data across 4MB objects. Was there a particular optimization or analysis that lead to this size?
[1:33] <AfC> (I probably have to ask this during the week)
[1:36] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[1:43] * mwarwick (~mwarwick@2407:7800:200:1011:3e97:eff:fe91:d9bf) has joined #ceph
[1:44] <bloodice> i keep breaking things lol
[1:45] <bloodice> holy.. its working haha.... four hours later
[1:46] * Fetch (fetch@gimel.cepheid.org) Quit (Ping timeout: 480 seconds)
[1:47] * nwat (~textual@eduroam-243-4.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[1:48] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[1:48] <bloodice> this is like a puzzle... gahh
[1:49] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[1:49] <bloodice> if i add stuff to the ceph.conf file, do i need to run the "sudo /etc/init.d/ceph -a restart" command?
[1:50] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[1:51] <pmatulis> bloodice: i never had too. why don't you conduct a test and see? you do need to inject the new file into the cluster though (should be the same on all nodes)
[1:52] <Nats_> AfC, i tested with smaller rbd blocks briefly and it performed a fair bit worse
[1:53] <bloodice> thats this right? "ceph-deploy config push monhost1 monhost2 monhost3"
[1:54] <bloodice> wait the way you said that.. does it need to every server in the cluster no matter what role?
[1:55] <pmatulis> prolly: ceph-deploy --overwrite-conf config push ...
[1:55] <pmatulis> yes, every node
[1:55] <bloodice> rgr
[1:56] <pmatulis> and client
[1:56] <bloodice> yea, i had to use --overwrite-conf when i added a monitor ( apparently required if it takes place after cluster creation )
[1:57] <pmatulis> i never needed to
[1:57] <pmatulis> i think you have the option of listing the new monitors ([mon] section i believe)
[1:58] <bloodice> lets see, i have another virtual to install and add a monitor
[1:59] * Cube (~Cube@66-87-65-18.pools.spcsdns.net) has joined #ceph
[2:04] * BillK (~BillK-OFT@106-68-78-193.dyn.iinet.net.au) Quit (Read error: Connection reset by peer)
[2:04] * BillK (~BillK-OFT@106-68-78-193.dyn.iinet.net.au) has joined #ceph
[2:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[2:12] <bloodice> yeap, you are correct, dont need --overwrite-conf ( it must of been because i had tried/failed several previous times on that host to create a monitor )
[2:12] <bloodice> monitor 3 came up right away :)
[2:12] * Cube (~Cube@66-87-65-18.pools.spcsdns.net) Quit (Quit: Leaving.)
[2:13] <pmatulis> bloodice: so no need to change ceph.conf at all right?
[2:13] <bloodice> well i updated the ceph.conf with the new monitor info before i installed it as a monitor
[2:14] <pmatulis> i never needed to do even that
[2:14] <bloodice> i did have to add the public network setting in the ceph.conf in order to even create a monitor(after initial install)
[2:14] <pmatulis> i did need to have declared my network, otherwise, my non-initial monitors would not get created properly
[2:15] <pmatulis> but no need to do anything after that
[2:15] <bloodice> yea that was the problem i was having before... but its all good now
[2:15] <bloodice> wait a sec... question
[2:15] <pmatulis> you should try adding a monitor without modifying ceph.conf (beyond the public network thing)
[2:15] <bloodice> oh ok
[2:16] <bloodice> dont you have to add it after though?
[2:16] <pmatulis> no
[2:16] <pmatulis> i'm using ceph-deploy, are you?
[2:16] <bloodice> yes i am
[2:16] <pmatulis> well, you might want to try it then
[2:16] <bloodice> does it add the entry automagically?
[2:16] <pmatulis> nope
[2:16] <kraken> http://i.imgur.com/xKYs9.gif
[2:17] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[2:17] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[2:17] <bloodice> so when you add a monitor, you dont add it to the ceph.conf on the admin node?
[2:17] <pmatulis> no
[2:18] <bloodice> are you saying that by adding it i am just doing a warm blanket type activity?
[2:18] <pmatulis> do you add OSDs to ceph.conf when you create new OSDs?
[2:18] <bloodice> i was
[2:19] <pmatulis> well, don't. you can, but i never did, so it's not necessary
[2:19] <bloodice> am i looking at old instructions?
[2:19] <bloodice> so if you reboot your cluster, it doesnt fall apart?
[2:19] <pmatulis> no
[2:19] <pmatulis> i presume ceph-deploy updates the MON map (and OSD map), dunno
[2:19] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[2:20] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[2:20] <pmatulis> if you do things manually (no ceph-deploy) then i do think you need to manually update ceph.conf
[2:20] <bloodice> that makes sense, cause editing a conf file is a pain... when all is done here, i will have like 180 drives
[2:20] <bloodice> i saw the manual instructions for host add then freaked out
[2:20] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit ()
[2:21] <bloodice> i then realized it was the manual task.. i really wish they would update the documentation to say "you dont need to do this if you use ceph-deploy"
[2:21] <bloodice> is there any way to view the map?
[2:21] <Nats_> ceph.conf doesnt need to be edited to add osd's
[2:22] <alfredodeza> bloodice: the docs are usually the other way around: "this is with ceph-deploy, but this other way is the manual one"
[2:22] <pmatulis> bloodice: what map?
[2:22] <bloodice> awesome... i was starting to dread the addition of this stuff
[2:22] <bloodice> the MON map
[2:22] <Nats_> ceph.conf lists the monitor addresses , the osd uses that to find a monitor, and asks monitor about other osd's
[2:23] <pmatulis> ceph mon dump
[2:23] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[2:23] <Nats_> so normally you'd only push a new ceph.conf to add a new monitor
[2:23] * sarob (~sarob@2601:9:7080:13a:45aa:9d8c:e325:f160) has joined #ceph
[2:23] <pmatulis> Nats_: it's not necessary
[2:23] <pmatulis> using ceph-deploy
[2:23] <alfredodeza> to add a new monitor there are a few steps though, not only ceph.conf
[2:24] <Nats_> true, ceph.conf on its own does nothing useful
[2:24] <pmatulis> alfredodeza: what did you think of my CEPH_DEPLOY key idea?
[2:25] <alfredodeza> I would *highly* recommend using `ceph-deploy mon create-initial` though if starting from scratch
[2:25] <alfredodeza> pmatulis: good. I somewhat missed out on thinking a user would want to specify a different key
[2:25] <alfredodeza> thanks for pointing that out
[2:26] <alfredodeza> we are starting to introduce other environment variables though, so it will probably be something like: CEPH_DEPLOY_SSH_KEY
[2:26] <pmatulis> ah ok
[2:27] <bloodice> alfredodeza: the directions are pretty good, though they are setup for people just getting up and running, you basically have to dig around to figure out what to do if you are doing stuff after the initial build ( its not a complaint though, just something i had to do )
[2:27] * glzhao (~glzhao@118.195.65.67) has joined #ceph
[2:27] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:28] <bloodice> so where are the OSDs mapped?
[2:28] <bloodice> they arent in the mon dump...
[2:28] * haomaiwang (~haomaiwan@117.79.232.175) Quit (Remote host closed the connection)
[2:30] <pmatulis> bloodice: why should they be
[2:30] <pmatulis> bloodice: you need to look at the OSD map
[2:30] <bloodice> i ran this and i get the info: "sudo ceph osd tree"
[2:31] <pmatulis> bloodice: well, let us know what you are actually looking for
[2:31] <pmatulis> bloodice: you asked for the MON map before
[2:31] <bloodice> yea, exactly
[2:31] * sarob (~sarob@2601:9:7080:13a:45aa:9d8c:e325:f160) Quit (Ping timeout: 480 seconds)
[2:31] <bloodice> well there are also entries for the osd's in ceph.conf file... i had been adding those
[2:32] <pmatulis> no need to
[2:32] <bloodice> so that command is the OSD dump command essentially?
[2:34] <pmatulis> i don't think so
[2:34] <bloodice> yea, if run "sudo ceph osd tree" it outputs the osd in the cluster and their locations
[2:34] <bloodice> and status
[2:35] <pmatulis> and where do you see locations in the OSD map?
[2:35] <bloodice> i mis-spoke on that.. its just the host name not the actual path
[2:36] <pmatulis> still, i don't see host names in the OSD map
[2:36] <pmatulis> the CRUSH map, however, has OSDs the hosts they're linked to
[2:36] <pmatulis> the CRUSH map, however, has OSDs and the hosts they're linked to
[2:36] <bloodice> oh boy i am burning up google on this stuff lol
[2:37] <pmatulis> don't go too fast or you'll go mad
[2:37] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[2:37] <bloodice> lol yea
[2:37] <bloodice> ok i will bite, do i have to dump that to a file to read it or is there a command to just print it on screen?
[2:38] <pmatulis> it's in binary form so you'll need to convert it to text and dump it to disk
[2:39] <bloodice> so i must ask the question.. do i really need to see it?
[2:39] <pmatulis> yes, if you want to learn ceph
[2:39] <pmatulis> in extreme situations you might need to edit it manually (convert, edit, reconvert, import)
[2:40] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[2:40] <pmatulis> crushtool -d crushmap.bin -o crushmap.txt
[2:40] <pmatulis> less crushmap.txt
[2:41] <bloodice> hehe i was doing those
[2:41] <bloodice> now i see!
[2:41] <pmatulis> great
[2:42] <bloodice> i havent even started messing with pools, mds, filesystem argh
[2:42] <bloodice> thanks!
[2:43] <bloodice> btw, i took your advice and created the virtual environment cause the physical servers were just too slow
[2:43] <pmatulis> i would concentrate on the basics, architecture, and so on. then get into rbd
[2:44] * Fetch (fetch@gimel.cepheid.org) has joined #ceph
[2:44] <bloodice> i was seriously sweating the config file... adding 190 osds manually.. ugh lol
[2:44] <pmatulis> tsk tsk
[2:44] <bloodice> i was about to break out the scripting
[2:46] * tsnider1 (~tsnider@198.95.226.40) Quit (Quit: Leaving.)
[2:49] * mschiff (~mschiff@port-3229.pppoe.wtnet.de) Quit (Remote host closed the connection)
[2:53] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[2:58] * rongze (~rongze@117.79.232.205) has joined #ceph
[3:04] * rongze (~rongze@117.79.232.205) Quit (Remote host closed the connection)
[3:05] * Fetch (fetch@gimel.cepheid.org) Quit (Read error: Connection reset by peer)
[3:08] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[3:10] * Fetch (fetch@gimel.cepheid.org) has joined #ceph
[3:13] * wenjianhn (~wenjianhn@123.118.211.50) has joined #ceph
[3:18] * The_Bishop (~bishop@2001:470:50b6:0:f48c:56c0:e6f7:14c5) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[3:23] * perfectsine (~perfectsi@if01-gn01.dal05.softlayer.com) Quit (Ping timeout: 480 seconds)
[3:24] * perfectsine (~perfectsi@if01-gn01.dal05.softlayer.com) has joined #ceph
[3:32] * rongze (~rongze@117.79.232.240) has joined #ceph
[3:33] * diegows (~diegows@190.190.11.42) Quit (Ping timeout: 480 seconds)
[3:34] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[3:35] <bloodice> did they add OSD removal to emperor or do i still need to use the manual process?
[3:38] <Nats_> still a painful manual process afaik
[3:38] <Nats_> see http://tracker.ceph.com/issues/3480
[3:38] <bloodice> k
[3:40] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[3:40] * Fetch (fetch@gimel.cepheid.org) Quit (Read error: Operation timed out)
[3:41] <bloodice> ok, so with a JBOD setup, if one of the disks fails, would you have to remove the OSD from the cluster or just replace the failed disk? My setup is to have the journal on the disk, so i am not sure how that would be able to repair if its dead.
[3:42] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[3:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[3:43] * rongze (~rongze@117.79.232.240) Quit (Ping timeout: 480 seconds)
[3:44] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[3:45] * Fetch (fetch@gimel.cepheid.org) has joined #ceph
[3:45] * alfredodeza (~alfredode@c-24-131-46-23.hsd1.ga.comcast.net) Quit (Remote host closed the connection)
[3:47] <Nats_> i believe you can replace a disk, but its still a bunch of manual commands
[3:47] <Nats_> like http://ceph.com/w/index.php?title=Replacing_a_failed_disk/OSD&redirect=no talks about
[3:48] <bloodice> nifty, i was looking at this one: http://eu.ceph.com/docs/wip-3060/ops/manage/failures/osd/ ( trying to figure out what to do when... )
[3:49] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[3:49] <Nats_> if you stick to the official docs, manual removal then add a new one via ceph-deploy seems to be the supported method
[3:51] <bloodice> with that, i would assume you would have to figure out what pool it was in and then add the new one to that pool
[3:52] <bloodice> assuming you have multiple pools ( havent dealt with pools yet, so guessing here )
[3:54] <bloodice> so if a host fails and you fix it, does that mean it can just rejoin as is or by rejoining you end up replicating all the data again anyways?
[3:55] * rongze (~rongze@117.79.232.204) has joined #ceph
[3:59] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[3:59] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[3:59] * smiley__ (~smiley@pool-108-28-107-254.washdc.fios.verizon.net) has joined #ceph
[4:03] * noahmehl_ (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[4:03] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[4:06] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Ping timeout: 480 seconds)
[4:06] * noahmehl_ is now known as noahmehl
[4:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[4:18] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[4:24] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:28] * rongze (~rongze@117.79.232.204) Quit (Remote host closed the connection)
[4:29] <bloodice> anyone know where i can get this? http://www.youtube.com/watch?v=Bmwl9d2b_Ic its some sort of visualization of ceph
[4:30] * rongze (~rongze@117.79.232.236) has joined #ceph
[4:32] <bloodice> found it.. https://github.com/rossturk/squidviz
[4:34] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[4:39] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Quit: noahmehl)
[4:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[4:50] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[4:56] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[4:57] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) has joined #ceph
[4:57] * anip_patel (~oftc-webi@ip72-201-57-180.ph.ph.cox.net) has joined #ceph
[4:58] * anip_patel (~oftc-webi@ip72-201-57-180.ph.ph.cox.net) has left #ceph
[5:01] * shang (~ShangWu@42.70.126.131) has joined #ceph
[5:02] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[5:06] * yy-nm (~Thunderbi@122.224.154.38) Quit (Read error: Connection reset by peer)
[5:09] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[5:11] * shang (~ShangWu@42.70.126.131) Quit (Remote host closed the connection)
[5:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[5:29] * torment3 (~torment@pool-71-180-185-11.tampfl.fios.verizon.net) has joined #ceph
[5:29] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[5:34] * torment2 (~torment@pool-72-64-241-72.tampfl.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[5:35] * yy-nm (~Thunderbi@122.224.154.38) has joined #ceph
[5:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[5:47] * rongze (~rongze@117.79.232.236) Quit (Remote host closed the connection)
[5:52] * Cube (~Cube@12.248.40.138) has joined #ceph
[5:54] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[6:04] * julian (~julianwa@125.70.135.211) has joined #ceph
[6:08] * JoeGruher (~JoeGruher@134.134.139.74) has joined #ceph
[6:09] <symmcom2> Hello, i am trying to create a new MON using ceph-deploy and it is giving me "ulimit -n 32768" error. Any idea why
[6:11] * sarob (~sarob@2601:9:7080:13a:dd20:f88f:6b84:718d) has joined #ceph
[6:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[6:16] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Quit: Leaving.)
[6:18] * rongze (~rongze@123.151.28.72) has joined #ceph
[6:20] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[6:23] * mtanski (~mtanski@cpe-74-65-252-48.nyc.res.rr.com) Quit (Quit: mtanski)
[6:29] * rongze (~rongze@123.151.28.72) Quit (Read error: Operation timed out)
[6:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[6:46] * topro_ (~tobi@ip-109-43-2-251.web.vodafone.de) Quit (Ping timeout: 480 seconds)
[6:53] * rongze (~rongze@117.79.232.236) has joined #ceph
[6:57] * sarob (~sarob@2601:9:7080:13a:dd20:f88f:6b84:718d) Quit (Remote host closed the connection)
[6:57] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[7:00] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[7:02] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[7:02] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[7:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[7:16] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[7:19] * JoeGruher (~JoeGruher@134.134.139.74) Quit (Remote host closed the connection)
[7:24] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Read error: Operation timed out)
[7:25] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[7:34] * AfC (~andrew@2407:7800:200:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[7:36] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[7:39] * smiley__ (~smiley@pool-108-28-107-254.washdc.fios.verizon.net) Quit (Quit: smiley__)
[7:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[7:48] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[7:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:07] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[8:07] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[8:08] * topro (~prousa@host-62-245-142-50.customer.m-online.net) has joined #ceph
[8:10] * mxmln (~mxmln@212.79.49.66) Quit (Quit: mxmln)
[8:11] * fladi (~fladische@sh144.medunigraz.at) has joined #ceph
[8:14] * rongze_ (~rongze@117.79.232.195) has joined #ceph
[8:18] * shimo (~A13032@122x212x216x66.ap122.ftth.ucom.ne.jp) has joined #ceph
[8:19] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[8:19] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[8:21] * rongze (~rongze@117.79.232.236) Quit (Ping timeout: 480 seconds)
[8:23] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[8:25] * tziOm (~bjornar@194.19.106.242) Quit (Remote host closed the connection)
[8:29] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[8:41] * dvanders (~dvanders@dvanders-air.cern.ch) has joined #ceph
[8:42] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[8:44] * rendar (~s@host23-191-dynamic.11-87-r.retail.telecomitalia.it) has joined #ceph
[8:47] * Sysadmin88 (~IceChat77@94.1.37.151) Quit (Quit: REALITY.SYS Corrupted: Re-boot universe? (Y/N/Q))
[8:54] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[9:00] * ksingh (~Adium@2001:708:10:10:2179:a576:40be:c299) has joined #ceph
[9:00] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[9:05] <ksingh> hello cephers
[9:06] <ksingh> anyone know command to find number of PG present in 1 pool ?
[9:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[9:14] <ksingh> dear all , any answers please ??
[9:16] <yanzheng> ceph osd pool get <poolname> pg_num
[9:17] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[9:18] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[9:18] * ChanServ sets mode +v andreask
[9:20] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[9:23] <ksingh> Thanks Yanzheng , and how to check size of 1 PG
[9:23] <yanzheng> no idea
[9:23] <ksingh> pool size ?
[9:26] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Ping timeout: 480 seconds)
[9:26] <yanzheng> ceph pg dump summary
[9:26] <yanzheng> ceph osd pool get poolname size
[9:27] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[9:28] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Ping timeout: 480 seconds)
[9:29] * mxmln (~mxmln@212.79.49.65) has joined #ceph
[9:38] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[9:42] * capri (~capri@212.218.127.222) Quit (Ping timeout: 480 seconds)
[9:43] * shang (~ShangWu@175.41.48.77) has joined #ceph
[9:44] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[9:46] * AfC (~andrew@2001:44b8:31cb:d400:2ad2:44ff:fe08:a4c) has joined #ceph
[9:47] * rongze (~rongze@117.79.232.168) has joined #ceph
[9:50] * rongze__ (~rongze@117.79.232.250) has joined #ceph
[9:51] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[9:53] * rongze_ (~rongze@117.79.232.195) Quit (Ping timeout: 480 seconds)
[9:54] * rongze__ (~rongze@117.79.232.250) Quit (Remote host closed the connection)
[9:55] * kainz (~anders@emp.sdu.dk) has joined #ceph
[9:57] * rongze (~rongze@117.79.232.168) Quit (Ping timeout: 480 seconds)
[9:59] * rongze (~rongze@211.155.113.236) has joined #ceph
[10:05] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) Quit (Quit: Leaving)
[10:06] * mattt_ (~textual@94.236.7.190) has joined #ceph
[10:07] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[10:10] * ScOut3R (~ScOut3R@catv-89-133-32-3.catv.broadband.hu) has joined #ceph
[10:12] * i_m (~ivan.miro@deibp9eh1--blueice2n2.emea.ibm.com) has joined #ceph
[10:14] * mwarwick (~mwarwick@2407:7800:200:1011:3e97:eff:fe91:d9bf) Quit (Quit: Leaving.)
[10:16] <BillK> Using 0.72: there is a "health HEALTH_WARN" in ceph -s but no detail as to what the problem is ... is there an easy way to find out?
[10:16] <andreask> can you pastebin the health -s output?
[10:16] <andreask> ceph -s ...
[10:18] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[10:18] * capri (~capri@212.218.127.222) has joined #ceph
[10:19] * capri (~capri@212.218.127.222) Quit ()
[10:19] <BillK> ceph health detail was what I needed ... getting low on OS disk space ... ceph -s didnt tell me what was the cause of the warning
[10:19] <BillK> tkx andreask
[10:27] * yy-nm (~Thunderbi@122.224.154.38) Quit (Quit: yy-nm)
[10:29] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[10:34] * smiley__ (~smiley@pool-108-28-107-254.washdc.fios.verizon.net) has joined #ceph
[10:36] * smiley__ (~smiley@pool-108-28-107-254.washdc.fios.verizon.net) Quit ()
[10:39] * LeaChim (~LeaChim@host86-162-2-255.range86-162.btcentralplus.com) has joined #ceph
[10:45] * mschiff (~mschiff@port-36097.pppoe.wtnet.de) has joined #ceph
[10:55] * mattt_ (~textual@94.236.7.190) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * AfC (~andrew@2001:44b8:31cb:d400:2ad2:44ff:fe08:a4c) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * mxmln (~mxmln@212.79.49.65) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * ksingh (~Adium@2001:708:10:10:2179:a576:40be:c299) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * julian (~julianwa@125.70.135.211) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * lurbs (user@uber.geek.nz) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * dwm (~dwm@northrend.tastycake.net) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * rtek (~sjaak@rxj.nl) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * gregorg (~Greg@78.155.152.6) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * plantain (~plantain@106.187.96.118) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * xmir (~xmeer@cm-84.208.159.149.getinternet.no) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * Pauline (~middelink@2001:838:3c1:1:be5f:f4ff:fe58:e04) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * madkiss (~madkiss@2001:6f8:12c3:f00f:fc7d:3d2d:3702:ca75) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * baffle (baffle@jump.stenstad.net) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * morse (~morse@supercomputing.univpm.it) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * cce (~cce@50.56.54.167) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * Knorrie (knorrie@yoshi.kantoor.mendix.nl) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * igalic (~hiro@panic.esotericsystems.at) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * vhasi (vhasi@vha.si) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * s15y (~s15y@sac91-2-88-163-166-69.fbx.proxad.net) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * raso (~raso@deb-multimedia.org) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * tnt_ (~tnt@ec2-54-200-98-43.us-west-2.compute.amazonaws.com) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * joao (~joao@a79-168-11-205.cpe.netcabo.pt) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * lxo (~aoliva@lxo.user.oftc.net) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * infernix (nix@cl-1404.ams-04.nl.sixxs.net) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * [fred] (fred@earthli.ng) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * alexxy (~alexxy@2001:470:1f14:106::2) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * yeled (~yeled@spodder.com) Quit (charon.oftc.net magnet.oftc.net)
[10:55] * vhasi (vhasi@vha.si) has joined #ceph
[10:55] * lurbs (user@uber.geek.nz) has joined #ceph
[10:55] * baffle (baffle@jump.stenstad.net) has joined #ceph
[10:55] * alexxy (~alexxy@2001:470:1f14:106::2) has joined #ceph
[10:55] * Knorrie (knorrie@yoshi.kantoor.mendix.nl) has joined #ceph
[10:55] * dwm (~dwm@northrend.tastycake.net) has joined #ceph
[10:55] * gregorg (~Greg@78.155.152.6) has joined #ceph
[10:55] * tnt_ (~tnt@ec2-54-200-98-43.us-west-2.compute.amazonaws.com) has joined #ceph
[10:55] * xmir (~xmeer@cm-84.208.159.149.getinternet.no) has joined #ceph
[10:55] * morse (~morse@supercomputing.univpm.it) has joined #ceph
[10:55] * cce (~cce@50.56.54.167) has joined #ceph
[10:55] * infernix (nix@cl-1404.ams-04.nl.sixxs.net) has joined #ceph
[10:55] * rtek (~sjaak@rxj.nl) has joined #ceph
[10:55] * madkiss (~madkiss@2001:6f8:12c3:f00f:cd93:9e3b:ed66:f8f8) has joined #ceph
[10:55] * mxmln (~mxmln@212.79.49.65) has joined #ceph
[10:55] * julian (~julianwa@125.70.135.211) has joined #ceph
[10:55] * Pauline (~middelink@2001:838:3c1:1:be5f:f4ff:fe58:e04) has joined #ceph
[10:55] * igalic (~hiro@panic.esotericsystems.at) has joined #ceph
[10:55] * ksingh (~Adium@2001:708:10:10:2179:a576:40be:c299) has joined #ceph
[10:55] * lxo (~aoliva@rainbowwarrior.torservers.net) has joined #ceph
[10:55] * plantain (~plantain@106.187.96.118) has joined #ceph
[10:55] * joao (~joao@a79-168-11-205.cpe.netcabo.pt) has joined #ceph
[10:55] * mattt_ (~textual@92.52.76.140) has joined #ceph
[10:55] * yeled (~yeled@spodder.com) has joined #ceph
[10:55] * s15y (~s15y@sac91-2-88-163-166-69.fbx.proxad.net) has joined #ceph
[10:55] * raso (~raso@deb-multimedia.org) has joined #ceph
[10:56] * AfC (~andrew@2001:44b8:31cb:d400:2ad2:44ff:fe08:a4c) has joined #ceph
[10:56] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) has joined #ceph
[10:56] * [fred] (fred@earthli.ng) has joined #ceph
[10:56] * ChanServ sets mode +o joao
[11:05] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) Quit (Ping timeout: 480 seconds)
[11:12] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:17] * hybrid512 (~walid@LPoitiers-156-86-25-85.w193-248.abo.wanadoo.fr) has joined #ceph
[11:19] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[11:21] * rongze (~rongze@211.155.113.236) Quit (Remote host closed the connection)
[11:24] * xdeller (~xdeller@91.218.144.129) has joined #ceph
[11:26] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[11:27] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Ping timeout: 480 seconds)
[11:29] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[11:29] * joshd (~joshd@2607:f298:a:607:c4fe:93eb:eb1a:c7fd) Quit (Ping timeout: 480 seconds)
[11:39] * joshd (~joshd@2607:f298:a:607:712e:7b76:e75c:ce84) has joined #ceph
[11:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[11:46] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[11:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[11:56] * fireD (~fireD@93-142-226-129.adsl.net.t-com.hr) has joined #ceph
[12:00] * nhm (~nhm@184-97-230-34.mpls.qwest.net) Quit (Ping timeout: 480 seconds)
[12:02] * shang (~ShangWu@175.41.48.77) Quit (Quit: Ex-Chat)
[12:09] * Siva_ (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[12:10] * gucki (~smuxi@77-56-39-154.dclient.hispeed.ch) has joined #ceph
[12:13] * dpippenger (~riven@cpe-75-85-17-224.socal.res.rr.com) has joined #ceph
[12:13] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Read error: Connection reset by peer)
[12:13] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[12:13] * ChanServ sets mode +v andreask
[12:13] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[12:14] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[12:14] * Siva_ is now known as Siva
[12:19] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[12:25] * jbd_ (~jbd_@2001:41d0:52:a00::77) has joined #ceph
[12:27] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Ping timeout: 480 seconds)
[12:31] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:36] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[12:43] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[12:46] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[12:47] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[12:49] * codice (~toodles@71-80-186-21.dhcp.lnbh.ca.charter.com) Quit (Remote host closed the connection)
[12:49] * codice (~toodles@71-80-186-21.dhcp.lnbh.ca.charter.com) has joined #ceph
[12:52] * alfredodeza (~alfredode@c-24-131-46-23.hsd1.ga.comcast.net) has joined #ceph
[12:52] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) Quit (Remote host closed the connection)
[12:55] * wenjianhn (~wenjianhn@123.118.211.50) Quit (Ping timeout: 480 seconds)
[12:56] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[12:56] * diegows (~diegows@190.190.11.42) has joined #ceph
[12:59] * hggh__ (~jonas@aljona.brachium-system.net) has left #ceph
[13:06] * mschiff (~mschiff@port-36097.pppoe.wtnet.de) Quit (Remote host closed the connection)
[13:08] * Cube1 (~Cube@12.248.40.138) has joined #ceph
[13:09] * jcsp (~Adium@0001bf3a.user.oftc.net) has joined #ceph
[13:09] * Cube (~Cube@12.248.40.138) Quit (Read error: Operation timed out)
[13:13] * Hakisho (~Hakisho@0001be3c.user.oftc.net) has joined #ceph
[13:14] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[13:14] * rongze (~rongze@117.79.232.205) has joined #ceph
[13:17] * t0rn (~ssullivan@69.167.130.11) has joined #ceph
[13:17] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[13:20] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[13:22] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[13:23] * i_m (~ivan.miro@deibp9eh1--blueice2n2.emea.ibm.com) Quit (Ping timeout: 480 seconds)
[13:23] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[13:24] * i_m (~ivan.miro@deibp9eh1--blueice4n2.emea.ibm.com) has joined #ceph
[13:24] * rongze (~rongze@117.79.232.205) Quit (Remote host closed the connection)
[13:25] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[13:27] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[13:28] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Ping timeout: 480 seconds)
[13:29] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[13:31] * AfC (~andrew@2001:44b8:31cb:d400:2ad2:44ff:fe08:a4c) Quit (Ping timeout: 480 seconds)
[13:32] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[13:34] * rongze_ (~rongze@117.79.232.205) has joined #ceph
[13:38] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[13:39] * alphe (~alphe@0001ac6f.user.oftc.net) has joined #ceph
[13:44] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[13:51] * alphe (~alphe@0001ac6f.user.oftc.net) Quit (Quit: Leaving)
[13:59] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[13:59] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:00] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[14:00] * ChanServ sets mode +v andreask
[14:01] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) has joined #ceph
[14:05] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) has joined #ceph
[14:11] * haomaiwang (~haomaiwan@211.155.113.163) has joined #ceph
[14:14] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:17] * mtk (~mtk@ool-44c35983.dyn.optonline.net) Quit (Remote host closed the connection)
[14:22] * mtk (~mtk@ool-44c35983.dyn.optonline.net) has joined #ceph
[14:23] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) Quit (Read error: Operation timed out)
[14:25] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:26] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[14:27] * allsystemsarego (~allsystem@188.26.167.226) has joined #ceph
[14:30] * BillK (~BillK-OFT@106-68-78-193.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[14:33] * smiley__ (~smiley@pool-108-28-107-254.washdc.fios.verizon.net) has joined #ceph
[14:33] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[14:37] * fretb (~fretb@frederik.pw) Quit (Quit: leaving)
[14:37] * fretb (~fretb@frederik.pw) has joined #ceph
[14:37] * tsnider (~tsnider@nat-216-240-30-23.netapp.com) has joined #ceph
[14:38] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[14:39] * fretb (~fretb@frederik.pw) Quit ()
[14:39] * fretb (~fretb@frederik.pw) has joined #ceph
[14:39] * david (~david@2001:4dd0:ff00:88c0:61d5:49b7:7d03:d7d) has joined #ceph
[14:43] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[14:43] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[14:43] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[14:44] * japuzzo (~japuzzo@ool-4570886e.dyn.optonline.net) has joined #ceph
[14:45] * david (~david@2001:4dd0:ff00:88c0:61d5:49b7:7d03:d7d) Quit (Remote host closed the connection)
[14:50] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:55] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) Quit (Remote host closed the connection)
[14:58] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) has joined #ceph
[15:02] * alaind (~dechorgna@161.105.182.35) has joined #ceph
[15:07] * loicd trying to understand how to move an osd to a specific location in the crush map
[15:07] <loicd> from the command line
[15:07] <loicd> hum
[15:08] <loicd> actually no, I want to move a host to a specific datacenter
[15:08] <ccourtaut> might be the same kind of command no, ie moving an item inside the crush map?
[15:08] <loicd> yes, but I don't know how to use them :-)
[15:09] * loicd reading about ceph crush
[15:10] <loicd> http://ceph.com/docs/master/rados/operations/crush-map/#add-move-an-osd
[15:13] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[15:14] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:14] * markbby (~Adium@168.94.245.2) has joined #ceph
[15:16] <loicd> ceph osd crush move bm0501 root=default datacenter=cloudwatt
[15:16] <loicd> just that ;-)
[15:18] <ccourtaut> oh, great
[15:18] <ccourtaut> :)
[15:20] <japuzzo> anyone play with the Vagrent build of ceph?
[15:22] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) Quit (Remote host closed the connection)
[15:27] <alfredodeza> there is a vagrant image around?
[15:27] <alfredodeza> I wasn't aware
[15:29] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:30] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[15:31] * Cube1 (~Cube@12.248.40.138) Quit (Quit: Leaving.)
[15:36] <japuzzo> alfredodeza, ya it's old though http://wiki.ceph.com/02Guides/Ceph_Vagrant_Setup
[15:36] * noahmehl_ (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[15:36] <japuzzo> Maybe an update can be my contribution
[15:37] <alfredodeza> interesting
[15:37] <alfredodeza> I think something a bit better might be to use a provider
[15:37] <alfredodeza> like ansible
[15:38] <alfredodeza> I did built one for ansible, all you need to do is to clone the repo, install ansible, and do `vagrant up`
[15:38] <alfredodeza> it will get you ceph installed iirc
[15:38] <alfredodeza> not to trump the wiki box, just an alternative
[15:38] * fladi (~fladische@sh144.medunigraz.at) Quit (Remote host closed the connection)
[15:39] <alfredodeza> japuzzo: https://github.com/alfredodeza/ceph-ansible
[15:40] <japuzzo> Well the one I posted is not working so I'm game thanks!
[15:41] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Ping timeout: 480 seconds)
[15:41] * noahmehl_ is now known as noahmehl
[15:41] <alfredodeza> japuzzo: let me know if you run into any trouble so I can update that
[15:42] <alfredodeza> it has been a while since I've posted updates :)
[15:42] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) has joined #ceph
[15:44] <japuzzo> alfredodeza, will do, have to look at ansible ( I'm working on a Windows laptop, want to be able to demo ceph otherwise it would be a Ubuntu server )
[15:44] <alfredodeza> it should not be a problem
[15:44] <alfredodeza> you only need python
[15:44] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[15:49] * yanzheng (~zhyan@jfdmzpr05-ext.jf.intel.com) Quit (Remote host closed the connection)
[15:50] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[15:51] <japuzzo> alfredodeza, Well I'm useing Cygwin which has 2.75 installed already wonder if that will work?
[15:51] <japuzzo> 2.7.5
[15:52] <alfredodeza> absolutely
[15:52] <alfredodeza> o/
[15:52] <kraken> \o
[15:52] <alfredodeza> :)
[15:52] * julian_ (~julianwa@125.70.135.211) has joined #ceph
[15:52] * SvenPHX1 (~Adium@wsip-174-79-34-244.ph.ph.cox.net) has joined #ceph
[15:53] * ScOut3R_ (~ScOut3R@catv-89-133-32-3.catv.broadband.hu) has joined #ceph
[15:53] * darkfaded (~floh@88.79.251.60) has joined #ceph
[15:53] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) Quit (Read error: Operation timed out)
[15:53] * darkfader (~floh@88.79.251.60) Quit (Read error: Connection reset by peer)
[15:53] * rendar (~s@host23-191-dynamic.11-87-r.retail.telecomitalia.it) Quit (Read error: Connection reset by peer)
[15:53] * noahmehl (~noahmehl@cpe-71-67-115-16.cinci.res.rr.com) has joined #ceph
[15:54] * The_Cog (~steve@cpc26-watf10-2-0-cust207.15-2.cable.virginm.net) has joined #ceph
[15:54] * The_Cog (~steve@cpc26-watf10-2-0-cust207.15-2.cable.virginm.net) has left #ceph
[15:54] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[15:54] * The_Cog (~steve@cpc26-watf10-2-0-cust207.15-2.cable.virginm.net) has joined #ceph
[15:54] * jdmason (~jon@134.134.137.71) Quit (Remote host closed the connection)
[15:54] * jonmason (~jon@192.55.54.41) has joined #ceph
[15:54] * SvenPHX (~Adium@wsip-174-79-34-244.ph.ph.cox.net) Quit (Read error: Operation timed out)
[15:55] * topro (~prousa@host-62-245-142-50.customer.m-online.net) Quit (Read error: Connection reset by peer)
[15:55] * topro (~prousa@host-62-245-142-50.customer.m-online.net) has joined #ceph
[15:55] * ScOut3R (~ScOut3R@catv-89-133-32-3.catv.broadband.hu) Quit (Read error: Operation timed out)
[15:55] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[15:58] * julian (~julianwa@125.70.135.211) Quit (Ping timeout: 480 seconds)
[16:04] <pmatulis> i've mostly read that all writes go to the primary OSD but i've also read that a secondary, tertiary, etc OSD can be "ahead" of the primary in terms of writes. how can this be? due to the primary be unavailable for some reason?
[16:05] <japuzzo> Seems harder then I thought ( http://www.quora.com/Christian-Nygaard/Posts/Using-Vagrant-and-Ansible-on-Windows )
[16:05] * jtang1 (~jtang@64.2.143.67.ptr.us.xo.net) Quit (Quit: Leaving.)
[16:06] * gregmark (~Adium@68.87.42.115) has joined #ceph
[16:06] <alfredodeza> argh
[16:07] <alfredodeza> japuzzo: this *might* not be what you want but if you have access to something non-windows you could then just use that VM?
[16:08] <japuzzo> ya, if there was any other way... no wifi or network so it has to be local
[16:08] <japuzzo> wonder if I can stack VM's and do it all from a Ubuntu VM
[16:08] <liiwi> yes
[16:09] * andreask (~andreask@zid-vpnn075.uibk.ac.at) has joined #ceph
[16:09] * ChanServ sets mode +v andreask
[16:10] * Siva (~sivat@117.192.35.0) has joined #ceph
[16:10] <japuzzo> so in the Ubuntu VM should I use openbox or kvm?
[16:12] * i_m (~ivan.miro@deibp9eh1--blueice4n2.emea.ibm.com) Quit (Quit: Leaving.)
[16:13] <pmatulis> heh?
[16:13] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[16:14] * Siva_ (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[16:18] * Siva (~sivat@117.192.35.0) Quit (Ping timeout: 480 seconds)
[16:18] * Siva_ is now known as Siva
[16:19] * diegows (~diegows@190.190.11.42) Quit (Ping timeout: 480 seconds)
[16:20] * Hakisho (~Hakisho@0001be3c.user.oftc.net) Quit (Ping timeout: 480 seconds)
[16:20] * jtang1 (~jtang@2001:468:1f07:da:b188:5cca:50f6:c272) has joined #ceph
[16:21] <madkiss> so if I want to deploy Ceph using Chef with external journals, I take it that defining the "journal" entry for every individual OSD is not sufficient?
[16:22] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[16:24] * jtang1 (~jtang@2001:468:1f07:da:b188:5cca:50f6:c272) Quit ()
[16:25] * jskinner (~jskinner@69.170.148.179) has joined #ceph
[16:26] <japuzzo> Quest, how come there isn't any offline docs for ceph? Like a PDF to read on a 5 hour plane trip?
[16:26] <japuzzo> Question more then a Quest 8-)
[16:28] <jcsp> japuzzo: just because nobody builds one, really. You could check out the ceph git repository and build yourself an offline html version in docs/
[16:29] <jcsp> it's sphinx, which has lots of output options
[16:29] * dpippenger (~riven@cpe-75-85-17-224.socal.res.rr.com) Quit (Quit: Leaving.)
[16:30] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[16:30] * cephn00b (~cephn00b@64.191.222.117) has joined #ceph
[16:30] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[16:31] <japuzzo> hummmm.....
[16:31] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[16:32] * Cube (~Cube@12.248.40.138) has joined #ceph
[16:33] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Read error: Connection reset by peer)
[16:33] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Remote host closed the connection)
[16:33] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[16:34] <cephn00b> The ceph guidelines recommend 1GB or RAM per TB of OSD. Does that number scale linearly as the amount of storage goes up? For instance, if I had a 200TB server, will I need 200GB RAM just for ceph?
[16:35] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[16:36] <Gugge-47527> a 200TB server?
[16:36] <Gugge-47527> Sounds like a bad plan for CEPH :)
[16:36] <Gugge-47527> That would be a lot of osd processes :)
[16:36] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[16:39] * themgt (~themgt@pc-188-95-160-190.cm.vtr.net) has joined #ceph
[16:42] <madkiss> well
[16:42] <madkiss> 200TB
[16:42] <madkiss> makes 50 disks per server. probably not a too great idea.
[16:44] * jtang1 (~jtang@wlan-rooms-226.sc13.org) has joined #ceph
[16:47] <alfredodeza> that is a lot of density for a single server
[16:48] <alfredodeza> wouldn't you want to have that storage more distributed?
[16:48] <janos> yeah that does not sound like a good match approach-wise for ceph
[16:48] <alfredodeza> I think it would be nicer to have more nodes
[16:48] <alfredodeza> not only for the RAM that was mentioned
[16:48] <janos> especially since the default failure domain is host
[16:48] <janos> that thing goes down....
[16:49] <janos> thrash-city
[16:49] <alfredodeza> exactly
[16:50] <alfredodeza> I think of nodes as hard drives in a server: more HDs can get you better redundancy just like more nodes
[16:54] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[16:56] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[16:57] <loicd> [ceph_deploy][ERROR ] UnsupportedPlatform: Platform is not supported: debian
[16:58] <loicd> alfredodeza: is there a way to tell ceph-deploy "do it anyway, I'm sure it will be fine" ?
[16:58] <alfredodeza> loicd: there is no way currently
[16:58] <alfredodeza> but
[16:58] <alfredodeza> that sounds like a bug
[16:59] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[16:59] <alfredodeza> and I think that was fixed recently
[16:59] <cephn00b> I am trying balance performance with density per rack. I really like the idea of 200TB in 4U.
[16:59] <alfredodeza> what version of ceph-deploy are you using loicd ?
[16:59] <loicd> alfredodeza: # ceph-deploy --version
[16:59] <loicd> 1.3.2
[16:59] <alfredodeza> hrmn
[16:59] <loicd> it's not ubuntu, it is debian wheezy
[16:59] <loicd> the target is debian wheezy
[16:59] <alfredodeza> that is so weird
[16:59] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: Textual IRC Client: www.textualapp.com)
[16:59] <loicd> but it's a very weird debian wheezy, don't sweat it ;-)
[17:00] <mattch> loicd: For redhat at least, I had to install some lsb packages before deph-deploy worked
[17:01] <loicd> I installed lsb-release
[17:01] <alfredodeza> we are no longer using lsb-release
[17:01] <alfredodeza> let me rephrase: ceph-deploy is no longer using lsb-release
[17:01] <alfredodeza> actually, the newest ceph-deploy versions don't have any dependencies
[17:02] <alfredodeza> I think since version 1.3
[17:02] <mattch> alredodoza: Ahh, good to know
[17:02] <loicd> oh
[17:02] <alfredodeza> it was such a pain before to keep up with all the tiny pieces that are needed so ceph-deploy can do some work
[17:02] * gdavis33 (~gdavis@66-87-117-4.pools.spcsdns.net) has joined #ceph
[17:02] <loicd> alfredodeza: what does it rely on then ?
[17:02] <alfredodeza> loicd: but you say it is a `weird debian wheezy` and I wonder why
[17:03] <alfredodeza> loicd: only Python :)
[17:03] <loicd> but how can it figure out it's debian ?
[17:03] <alfredodeza> loicd: Python has a built in module to detect that, let me get you a command so you can tell me the output
[17:03] <alfredodeza> one sec
[17:04] <dmsimard> Anyone from inktank around to answer questions regarding the ceph training http://www.inktank.com/ceph130-unified-storage-for-openstack/ ?
[17:04] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[17:05] * jtang1 (~jtang@wlan-rooms-226.sc13.org) Quit (Read error: Operation timed out)
[17:05] <alfredodeza> loicd: can you run this (including quotes) in that server? --> python -c "import platform; print platform.linux_distribution()"
[17:05] <loicd> python -c "import platform; print platform.linux_distribution()"
[17:05] <loicd> ('debian', 'wheezy/sid', '')
[17:05] <alfredodeza> oh my god
[17:06] <alfredodeza> wheezy/sid
[17:06] <alfredodeza> really?
[17:06] <loicd> told ya
[17:06] <loicd> ahahah
[17:06] * loicd trying to hack this into something less weird
[17:07] <alfredodeza> the reason it needs that is because one of the things ceph-deploy does is to add the right lines to sources.list.d/ceph.list
[17:07] <alfredodeza> so it needs the right release name
[17:08] <alfredodeza> otherwise I am not sure this ceph.list would work: deb {url} wheezy/sid main
[17:08] <loicd> true
[17:08] <loicd> where is the list of supported os ?
[17:09] <alfredodeza> good question
[17:10] <alfredodeza> I am not sure if there is one, but from ceph-deploy: https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/hosts/__init__.py#L57-64
[17:10] <alfredodeza> loicd: ^ ^
[17:10] * loicd looking
[17:14] * allsystemsarego_ (~allsystem@5-12-240-115.residential.rdsnet.ro) has joined #ceph
[17:14] * allsystemsarego (~allsystem@188.26.167.226) Quit (Read error: Connection reset by peer)
[17:14] <loicd> this is weird, it says "debian" not supported but it is among the supported platforms
[17:15] <loicd> oh, is it the case that running ceph-deploy from a ubuntu platform with a target being a debian wheezy is going to trigger that ?
[17:15] <KevinPerks> dmsimard: I can answer questions on the training
[17:16] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[17:17] * allsystemsarego_ (~allsystem@5-12-240-115.residential.rdsnet.ro) Quit ()
[17:17] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[17:17] <loicd> I get it
[17:18] <alfredodeza> loicd: mmmmn, the detection is done remotely
[17:19] <alfredodeza> on the host that is to get ceph installed
[17:19] <loicd> yes, and the missing codename on my weird distro is missing
[17:19] <alfredodeza> right
[17:19] <loicd> I'll hack it away
[17:19] <alfredodeza> sorry :(
[17:19] <loicd> thanks for the help. It's my fault really, this box is a mess ;-)
[17:28] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[17:29] * ksingh (~Adium@2001:708:10:10:2179:a576:40be:c299) Quit (Ping timeout: 480 seconds)
[17:30] * gdavis33 (~gdavis@66-87-117-4.pools.spcsdns.net) has left #ceph
[17:31] * mattt_ (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[17:32] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[17:34] * mattt_ (~textual@94.236.7.190) has joined #ceph
[17:35] * DarkAceZ (~BillyMays@50.107.53.200) has joined #ceph
[17:37] * thomnico (~thomnico@128.65.89.71) has joined #ceph
[17:38] * ircolle (~Adium@2001:468:1f07:da:c08a:e509:92c8:3d8) has joined #ceph
[17:39] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[17:40] * mattt__ (~textual@92.52.76.140) has joined #ceph
[17:41] * gdavis331 (~gdavis@38.122.12.254) has joined #ceph
[17:42] * mattt_ (~textual@94.236.7.190) Quit (Ping timeout: 480 seconds)
[17:42] * gdavis331 (~gdavis@38.122.12.254) Quit ()
[17:43] * Shmouel (~Sam@fny94-12-83-157-27-95.fbx.proxad.net) Quit (Quit: Leaving.)
[17:44] * DarkAceZ (~BillyMays@50.107.53.200) Quit (Ping timeout: 480 seconds)
[17:46] * xarses (~andreww@64-79-127-122.static.wiline.com) has joined #ceph
[17:48] * DarkAceZ (~BillyMays@50.107.53.200) has joined #ceph
[17:48] * mattt__ (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[17:50] * gucki (~smuxi@77-56-39-154.dclient.hispeed.ch) Quit (Remote host closed the connection)
[17:51] * zoltan (~zoltan@2001:620:20:16:b565:9e49:5de0:2afd) has joined #ceph
[17:51] <zoltan> hey
[17:51] * jskinner (~jskinner@69.170.148.179) Quit (Remote host closed the connection)
[17:52] <zoltan> so originally I had one box to use ceph on so I had set osd_crush_chooseleaf_type=0
[17:52] <zoltan> now I'm thinking about removing some OSDs and moving them to an other physical box
[17:52] <zoltan> is it enough if I remove this from the config? how could I force a rebalance?
[17:55] <zoltan> and for removal, should I first do a ceph osd out, then a down, and then remove it from the crush map?
[17:56] * glzhao (~glzhao@118.195.65.67) Quit (Quit: leaving)
[17:59] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[17:59] * gdavis33 (~gdavis@38.122.12.254) has joined #ceph
[17:59] * gdavis33 (~gdavis@38.122.12.254) has left #ceph
[17:59] * mxmln (~mxmln@212.79.49.65) Quit (Quit: mxmln)
[18:00] * ircolle (~Adium@2001:468:1f07:da:c08a:e509:92c8:3d8) Quit (Quit: Leaving.)
[18:02] * Sysadmin88 (~IceChat77@94.1.37.151) has joined #ceph
[18:04] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[18:05] * ScOut3R_ (~ScOut3R@catv-89-133-32-3.catv.broadband.hu) Quit (Ping timeout: 480 seconds)
[18:05] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[18:05] * The_Cog (~steve@cpc26-watf10-2-0-cust207.15-2.cable.virginm.net) has left #ceph
[18:07] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Read error: Connection reset by peer)
[18:07] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Read error: Connection reset by peer)
[18:07] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[18:07] * eternaleye_ is now known as eternaleye
[18:07] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[18:08] * andreask (~andreask@zid-vpnn075.uibk.ac.at) Quit (Ping timeout: 480 seconds)
[18:10] * aliguori (~anthony@74.202.210.82) has joined #ceph
[18:10] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[18:15] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has left #ceph
[18:16] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[18:16] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) Quit (Read error: Connection reset by peer)
[18:16] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[18:20] * alaind (~dechorgna@161.105.182.35) Quit (Ping timeout: 480 seconds)
[18:21] * mtanski (~mtanski@69.193.178.202) Quit (Quit: mtanski)
[18:26] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:27] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[18:31] * nwat (~textual@eduroam-243-4.ucsc.edu) has joined #ceph
[18:37] * ircolle (~Adium@2001:468:1f07:da:3d4f:1e3e:aa4a:8413) has joined #ceph
[18:38] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[18:38] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[18:39] * rendar (~s@host23-191-dynamic.11-87-r.retail.telecomitalia.it) has joined #ceph
[18:39] * bandrus (~Adium@c-98-238-148-252.hsd1.ca.comcast.net) has joined #ceph
[18:40] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[18:40] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[18:43] <zoltan> if I do an osd out, and then a crush remove, why does rebalancing happens twice?
[18:47] * cephn00b (~cephn00b@64.191.222.117) Quit (Quit: cephn00b)
[18:47] * gregsfortytwo (~Adium@2607:f298:a:607:2563:b849:a715:1421) Quit (Quit: Leaving.)
[18:47] * gregsfortytwo (~Adium@2607:f298:a:607:f428:d81b:4739:7126) has joined #ceph
[18:50] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[18:51] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[18:52] * jskinner (~jskinner@69.170.148.179) has joined #ceph
[18:54] * diegows (~diegows@mail.bittanimation.com) has joined #ceph
[18:55] * thomnico (~thomnico@128.65.89.71) Quit (Ping timeout: 480 seconds)
[18:56] <xdeller> zoltan: it is a known limitation from current data placement logic
[18:57] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[18:57] <xdeller> I hope it`ll be fixed soon and if you will create ticket it would be awesome
[18:59] * alram (~alram@38.122.20.226) has joined #ceph
[18:59] <zoltan> sure
[18:59] * cephn00b (~cephn00b@64.191.222.117) has joined #ceph
[19:00] <zoltan> do you have an answer for my first question too? :) how can I rebalance my cluster from just one node to >1?
[19:00] * jskinner (~jskinner@69.170.148.179) Quit (Ping timeout: 480 seconds)
[19:01] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[19:01] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Read error: Connection reset by peer)
[19:01] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) Quit (Remote host closed the connection)
[19:02] * eternaleye (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[19:02] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[19:03] * cephn00b (~cephn00b@64.191.222.117) Quit ()
[19:04] <xdeller> not sure if understood you correctly, you want to move all your data to other box saving rf=1?
[19:04] <kraken> ???_???
[19:04] * angdraug (~angdraug@64-79-127-122.static.wiline.com) has joined #ceph
[19:04] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[19:05] * eternaleye_ (~eternaley@c-24-17-202-252.hsd1.wa.comcast.net) has joined #ceph
[19:07] * sarob (~sarob@nat-dip12.cfw-a-gci.corp.yahoo.com) has joined #ceph
[19:08] * JoeGruher (~JoeGruher@jfdmzpr04-ext.jf.intel.com) has joined #ceph
[19:08] * linuxkidd (~linuxkidd@2607:f298:a:607:e8b:fdff:fe5a:47c7) has joined #ceph
[19:09] * Sysadmin88 (~IceChat77@94.1.37.151) Quit (Quit: Man who run behind car get exhausted)
[19:14] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[19:15] * unit3 (~Unit3@72.2.49.50) has joined #ceph
[19:17] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[19:23] * linuxkidd (~linuxkidd@2607:f298:a:607:e8b:fdff:fe5a:47c7) Quit (Quit: Konversation terminated!)
[19:32] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[19:32] * scuttlemonkey (~scuttlemo@2601:1:bf00:dda:7dbb:d517:f399:25a4) has joined #ceph
[19:32] * ChanServ sets mode +o scuttlemonkey
[19:32] <unit3> So, my primary MDS failed last week while writing data, and now it's pretty messed up. I just get this when I try to start it: mds.0.journaler(ro) error getting journal off disk
[19:33] <unit3> and then it dies.
[19:33] <unit3> ceph says all of the objects in pools are healthy, nothing's missing.
[19:33] <unit3> does anyone have any suggestions on how to repair this?
[19:34] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[19:36] * jskinner (~jskinner@69.170.148.179) has joined #ceph
[19:37] * cjh973 (~cjh973@ps123903.dreamhost.com) has joined #ceph
[19:38] * zoltan (~zoltan@2001:620:20:16:b565:9e49:5de0:2afd) Quit (Quit: Leaving)
[19:38] * linuxkidd (~linuxkidd@2607:f298:a:607:e8b:fdff:fe5a:47c7) has joined #ceph
[19:38] <cjh973> i'm not sure if my email every made it to the dev list
[19:39] <cjh973> anyone see a message on the mailing list about an MDS server going into a boot loop?
[19:40] <mikedawson> cjh973: i see the message
[19:42] <cjh973> oh awesome
[19:42] <cjh973> ok
[19:43] <cjh973> yeah every few days i notice my mds-a server going insane and just endlessly looping
[19:43] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[19:45] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[19:45] * ChanServ sets mode +v andreask
[19:48] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[19:51] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[19:51] <cjh973> is it just me or does the ceph channel seem quieter these days?
[19:53] * scuttlemonkey (~scuttlemo@2601:1:bf00:dda:7dbb:d517:f399:25a4) Quit (Read error: Connection reset by peer)
[19:53] * astark (~astark@6cb32e01.cst.lightpath.net) Quit ()
[19:53] * scuttlemonkey (~scuttlemo@c-174-51-178-5.hsd1.co.comcast.net) has joined #ceph
[19:53] * ChanServ sets mode +o scuttlemonkey
[19:54] <unit3> yeah, it was definitely more chatty a few weeks ago.
[19:55] * dmick (~dmick@2607:f298:a:607:657c:6f08:b568:2b60) Quit (Quit: Leaving.)
[19:55] <cjh973> maybe everyone is gearing up for the holidays :)
[19:55] * dmick (~dmick@2607:f298:a:607:e5a2:368c:aef0:ff0d) has joined #ceph
[19:56] <mikedawson> Dumpling is quite solid, Emperor was released in much better shape than the previous named releases, ceph-deploy is getting better, lots of Inktank staffers have been traveling to conventions, Sage is on vacation
[19:57] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[19:57] <mikedawson> so it is oddly quite
[19:57] <mikedawson> quiet
[19:57] <cjh973> yeah i like dumpling
[19:57] * ircolle (~Adium@2001:468:1f07:da:3d4f:1e3e:aa4a:8413) Quit (Quit: Leaving.)
[19:57] <cjh973> i'm a little nervous to upgrade to emperor
[19:57] <cjh973> but i use cephfs really heavily so any patches are highly appreciated
[19:59] <cjh973> mikedawson: think emperor is solid enough to upgrade to?
[19:59] <unit3> My upgrade to emperor was smooth, but then I had this MDS failure that I've yet to figure out.
[19:59] <cjh973> oh :(
[19:59] <cjh973> mds failures would really screw me up bad
[19:59] <cjh973> that's like my entire workload
[20:01] * sarob (~sarob@nat-dip12.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[20:01] <mikedawson> cjh973: I don't know of any show-stoppers, but I am still testing in my development lab. I'll likely won't upgrade this week
[20:01] * sarob (~sarob@nat-dip12.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:02] <mikedawson> i need the time to work on my english skills, apparently
[20:03] * sarob (~sarob@nat-dip12.cfw-a-gci.corp.yahoo.com) Quit (Read error: Operation timed out)
[20:04] <dmick> :)@mike
[20:04] <unit3> cjh973: mine was due to a hardware failure, not due to anything in hardware I think. but I don't know enough about how the mds works to recover it.
[20:04] <unit3> erm not due to anything in software. :P
[20:04] <unit3> still waking up, apparently. :P
[20:05] <mikedawson> dmick: i have a bad habit of hitting enter, then reading they rubbish I just typed
[20:05] <dmick> ^ case in point :) and me too
[20:06] <mikedawson> ha
[20:06] <cjh973> unit3: ok cool. maybe i'll take the plunge this weekend
[20:07] <unit3> and if you've got advice on troubleshooting mds issues, or some good docs to look at, I'd appreciate the pointer. Mine's now complaining it can't get the journal off disk, and then just dies, even though ceph says all objects are fine.
[20:10] <cjh973> unit3: i'm no expert on the mds. usually it just works
[20:10] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[20:11] <unit3> haha yeah. I was just unlucky and had the active mds die while I was writing a crapload of data, and now it seems really unhappy. I think if I can just reset the journal it'll be fine, but I haven't figured out how to do that.
[20:12] <cjh973> oh i see
[20:13] <cjh973> might want to start digging through the code or the dev docs
[20:14] * smiley__ (~smiley@pool-108-28-107-254.washdc.fios.verizon.net) Quit (Quit: smiley__)
[20:15] <unit3> well, it's not in the wiki docs, but -h in the latest release shows there's a --reset-journal <rank>, so I'm trying that.
[20:15] <unit3> It's just sitting there, not logging anything, doesn't really seem to be doing much. but it hasn't died, so I'll just let it run.
[20:16] <cjh973> yeah no news is good news i suppose
[20:19] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has left #ceph
[20:19] <unit3> I wonder how long this is supposed to take? too bad there's no devs around to advise. it's showing as up, but stuck in "replay", and I still can't mount the fs.
[20:20] * dxd828 (~dxd828@host-92-24-127-29.ppp.as43234.net) has joined #ceph
[20:22] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:22] * xdeller (~xdeller@91.218.144.129) Quit (Quit: Leaving)
[20:23] * davidz (~Adium@ip68-5-239-214.oc.oc.cox.net) has joined #ceph
[20:28] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:29] <unit3> ahhh debug levels are my friend. this is what I get when trying to start up the mds: 2013-11-18 11:28:54.601546 7f13bf65f700 1 -- 172.17.2.21:6805/19874 <== osd.2 172.17.2.23:6800/29447 2 ==== osd_op_reply(5 200.00000000 [read 0~0] v0'0 uv0 ack = -23 (Too many open files in system)) v6 ==== 171+0+0 (2033837461 0 0) 0x3114a00 con 0x30c3fa0
[20:29] * jbd_ (~jbd_@2001:41d0:52:a00::77) has left #ceph
[20:30] <dmick> unit3: that sounds a lot like the problem we just released a .1 for
[20:30] <unit3> I thought I upgraded to .1, but I maybe didn't restart all my services.
[20:30] <unit3> I'll try that now.
[20:30] <dmick> ceph tell osd.* version
[20:30] <dmick> although I guess that won't help for a nonstarted osd
[20:34] * mtanski (~mtanski@69.193.178.202) Quit (Quit: mtanski)
[20:34] <unit3> well, I just cycled all my osds, they're definitely all running from the 0.72.1 package, but I'm still getting that error right before I get the one about not being able to grab the journal.
[20:34] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[20:35] <unit3> and tell version confirms they're all that release, as well.
[20:35] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[20:35] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[20:37] * thomnico (~thomnico@81.253.41.52) has joined #ceph
[20:38] <unit3> and the OSD which is reporting that error only has ~170 open fds according to the kernel, so that's well below the default limit of 1024.
[20:39] <unit3> dmick: do you have a link to the bug report about this issue? I'm curious to see the details of the one that was fixed in .1, see how it relates to what I'm seeing.
[20:42] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) Quit (Remote host closed the connection)
[20:43] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) has joined #ceph
[20:43] * symmcom2 (~symmcom@184.70.203.22) Quit (Ping timeout: 480 seconds)
[20:45] <unit3> cjh973: so if this is an issue related to the emperor upgrade, I guess you'll probably wanna hold off. ;)
[20:46] <cjh973> yeah i need my fuse mount to keep humming
[20:48] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[20:50] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[20:50] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:51] * mxmln (~mxmln@212.79.49.66) has joined #ceph
[20:51] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) Quit (Remote host closed the connection)
[20:51] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) has joined #ceph
[20:51] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[20:52] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:52] * sarob (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[20:52] <unit3> unfortunately, --reset-journal doesn't seem to ever do anything.
[20:52] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:54] <unit3> ahhh, release notes have a repair tool: http://ceph.com/docs/master/release-notes/#v0-72-1-emperor
[20:55] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) Quit ()
[20:56] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) has joined #ceph
[20:57] <cjh973> unit3: that's good :)
[20:57] <unit3> oh yeah, that repair tool is finding tons of stuff to fix.
[20:58] <cjh973> awesome
[20:58] <unit3> yep, a good sign for sure.
[20:59] <cjh973> so maybe the upgrade is back on ? :)
[20:59] * Hakisho (~Hakisho@0001be3c.user.oftc.net) has joined #ceph
[21:00] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) Quit ()
[21:00] <unit3> could be! I'll let you know once I'm done this repair step, if it comes back up ok.
[21:00] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) has joined #ceph
[21:00] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[21:02] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[21:02] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[21:02] <unit3> yeah! mdses are up, cephfs is mounted. :)
[21:02] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[21:02] <unit3> so it sounds like if you go right to 0.72.1 you should be able to skip this little problem.
[21:03] * astark (~astark@6cb32e01.cst.lightpath.net) has joined #ceph
[21:03] <unit3> good job ceph devs! that was a very prompt bugfix, and the repair tool worked great!
[21:03] <unit3> now to continue syncing data onto my fs.
[21:05] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) Quit ()
[21:05] * erwan_taf (~erwan@lns-bzn-48f-62-147-157-222.adsl.proxad.net) has joined #ceph
[21:07] <cjh973> sweet
[21:07] <cjh973> ok 0.72.1 it is
[21:10] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[21:10] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[21:12] * scuttlemonkey (~scuttlemo@c-174-51-178-5.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[21:19] * thomnico (~thomnico@81.253.41.52) Quit (Quit: Ex-Chat)
[21:26] * astark (~astark@6cb32e01.cst.lightpath.net) Quit (Quit: astark)
[21:27] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:32] <dmick> unit3: thumbs up
[21:32] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[21:33] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[21:35] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[21:39] * jefferai (~quassel@corkblock.jefferai.org) has joined #ceph
[21:39] <cjh973> unit3: my mds.a server is stuck in that boot loop again
[21:39] <cjh973> ugh
[21:44] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[21:45] * bandrus (~Adium@c-98-238-148-252.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[21:47] <cjh973> i keep seeing this over and over in my ceph logs: 2013-11-18 15:46:57.661689 mon.0 [INF] mds.? 192.168.1.21:6800/1280 up:boot
[21:47] <cjh973> if i kill -9 the process it says reconnecting and then it's fine
[21:51] <jhujhiti> 'ceph osd out 3' => 'osd.3 is already out', but 'ceph osd rm 3' => 'osd.3 is still up'
[21:51] <jhujhiti> what?
[21:52] <janos> in/out up/down
[21:52] <janos> make sure it's down
[21:52] <jhujhiti> well the process is dead so it's sure as hell down
[21:52] <janos> sounds like it
[21:52] <jhujhiti> but it is still showing up in ceph osd dump
[21:52] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[21:53] <janos> what does an init.d/ceph stop osd.3 do?
[21:54] <jhujhiti> /etc/init.d/ceph: osd.3 not found (/etc/ceph/ceph.conf defines "")
[21:54] <jhujhiti> i have no idea how this was starting - it was using upstart
[21:54] <janos> ah
[21:54] <jhujhiti> i was able to stop another osd on the same machine
[21:55] <jhujhiti> and remove it, i should say
[21:55] <jhujhiti> odd
[21:56] * rovar (~rovar@pool-108-6-176-201.nycmny.fios.verizon.net) has joined #ceph
[21:56] * dmick (~dmick@2607:f298:a:607:e5a2:368c:aef0:ff0d) Quit (Quit: Leaving.)
[21:56] * dmick (~dmick@2607:f298:a:607:e5a2:368c:aef0:ff0d) has joined #ceph
[21:59] * Sysadmin88 (~IceChat77@94.1.37.151) has joined #ceph
[21:59] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) has joined #ceph
[21:59] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[22:02] <cjh973> unit3: does your ceph -w show your mds server changing ports once per second?
[22:10] <japuzzo> Is there anyone from Inktank at Supercomputing in Denver?
[22:10] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[22:10] <jhujhiti> so how do i force the mons to figure out that these osds are down?
[22:11] <dmick> japuzzo: yes
[22:12] <japuzzo> dmick, so of my associates are also there and would be open to meet
[22:14] <dmick> I know there's at least one Inktank demo on the show floor; I don't know whether it's our booth or a partner's
[22:14] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[22:17] <unit3> cjh973: nope, I'm not seeing that kind of mds loop you are.
[22:18] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:20] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[22:20] <hjorth> Anybody got any idea, why cephfs started acting weird after i added a 3rd OSD (and dividing into 2 different data centers)? By acting weird, it basically means that its very long time about writing data.
[22:21] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) has joined #ceph
[22:23] <unit3> It's probably trying to do data replication to the 2nd data center, I'd assume. and that'll incure heavy latency.
[22:24] <cjh973> hjorth: you have to get an ack from all osd's before it'll ack back to the client
[22:26] <hjorth> My ping time is 1ms if that means anything..
[22:26] <hjorth> Its sync data prerry fast when doing recovery and remapping data.
[22:27] * KindTwo (KindOne@h189.208.89.75.dynamic.ip.windstream.net) has joined #ceph
[22:27] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[22:27] * KindTwo is now known as KindOne
[22:27] <unit3> you might want to pastebin your crush dump so people can take a look. might illuminate what's going on, since it sounds like your intersite link is pretty fast.
[22:28] <mikedawson> hjorth: to eliminate guessing, do 'ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops'. That will show you the 20 slowest operations in the last 10 mins (unless you changed settings). If all your ops are fast, look somewhere else.
[22:28] * linuxkidd (~linuxkidd@2607:f298:a:607:e8b:fdff:fe5a:47c7) Quit (Remote host closed the connection)
[22:28] * linuxkidd (~linuxkidd@2607:f298:a:607:9eeb:e8ff:fe07:6658) has joined #ceph
[22:29] <hjorth> http://pastebin.com/PK96RQ8F
[22:31] <mikedawson> hjorth: it seems like the datacenter taastrup is not listed in root default
[22:31] <mikedawson> are you trying to replicate between the two or use them separately?
[22:32] <hjorth> mikedawson: nope, testing like an example on ceph.com.. Its the same picture when adding datacenter taastrup to root default.
[22:32] <hjorth> basically i am trying to make a setup, where there should be 2 copies of all object, one in each datacenter..
[22:33] <jhujhiti> hjorth: take -1 first, then take 1
[22:33] <jhujhiti> maybe? i just did something similar and that's how i got it to work
[22:33] <hjorth> in the historic_ops the longest duration is 2.240250
[22:34] <mikedawson> hjorth: unless I'm not understanding something, I would add taastrup to default, then make you rule be step chooseleaf firstn 0 type datacenter
[22:35] <hjorth> mikedawson: I tried that, but experienced the same.. But i am more then happy the test again..
[22:36] <mikedawson> hjorth: 2.2 seconds is quite high, but without knowing more about latencies, it is tough to tell if *everything* is slow, or just a small set of outliers are that high
[22:37] <mikedawson> hjorth: the way to tell is to sample 'ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok perf dump' and look for sub_op latency sum divided by avgcount
[22:37] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[22:38] * astark (~astark@ool-4576c894.dyn.optonline.net) has joined #ceph
[22:38] <hjorth> mikedawson: Okey.. it btw seems like that cephfs is the only one affected.. I have "fine" throughput with s3/rgw.
[22:39] <mikedawson> hjorth: if you sample that data (and journal latencies), every 10s, then graph, you'll get something like this http://www.gammacode.com/subop-and-journal-latency.jpg
[22:39] <mikedawson> very useful in tracking down performance issues
[22:41] <hjorth> and i should do this on the OSD which seem to be the slow one, right?
[22:42] <hjorth> "subop_latency": { "avgcount": 25539,"sum": 10103.182845000}, = 10103/25539, this is the number right?
[22:42] <mikedawson> hjorth: i do it across *all* osds. If you have a large number of OSDs, it is nearly impossible to tell which OSD is slow because of Ceph's psuedo-random placement. This method helps.
[22:44] <mikedawson> hjorth: yes that's the right data. Also look at journal_latency. But sampling over time will show you what you want. You basically have to do a time-adjusted derivative in Graphite terms.
[22:44] * astark (~astark@ool-4576c894.dyn.optonline.net) Quit (Quit: astark)
[22:45] <mikedawson> hjorth: are your 3 osds backed with RAID sets, or bare drives?
[22:45] * BillK (~BillK-OFT@106-68-78-193.dyn.iinet.net.au) has joined #ceph
[22:45] <via> is there a problem with the gpg key for leveldb in the el6/emperor repo?
[22:46] <hjorth> mikedawson: Its actually virtual machines..
[22:47] <hjorth> mikedawson: the osd 0+1 running on local raid in an esx host... osd 2, running on a SAN installation with iscsi.
[22:48] <hjorth> I am still in the proof of concept.
[22:48] <hjorth> But osd 0+1 runs very well.
[22:49] <mikedawson> hjorth: ok. you'll probably get poor to really poor performance. for production, you'll want lots of spindles or ssds and >3 osds
[22:50] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) Quit (Quit: ChatZilla 0.9.90.1 [Firefox 25.0.1/20131112160018])
[22:50] <hjorth> mikedawson: yes i am aware of that.. But there is big difference between poor performance and an experience of something not working..
[22:54] * LPG (~LPG@c-76-104-197-224.hsd1.wa.comcast.net) has joined #ceph
[22:59] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:00] * yanzheng (~zhyan@134.134.139.74) has joined #ceph
[23:04] * onizo (~onizo@wsip-70-166-5-159.sd.sd.cox.net) has joined #ceph
[23:09] * diegows (~diegows@mail.bittanimation.com) Quit (Ping timeout: 480 seconds)
[23:10] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:14] * mtanski (~mtanski@69.193.178.202) Quit (Ping timeout: 480 seconds)
[23:22] * sarob_ (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) has joined #ceph
[23:24] * linuxkidd (~linuxkidd@2607:f298:a:607:9eeb:e8ff:fe07:6658) Quit (Remote host closed the connection)
[23:24] * linuxkidd (~linuxkidd@2607:f298:a:607:e8b:fdff:fe5a:47c7) has joined #ceph
[23:24] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:28] * sjm (~sjm@pool-96-234-124-66.nwrknj.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[23:29] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[23:30] * sarob_ (~sarob@nat-dip28-wl-b.cfw-a-gci.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[23:33] * alram (~alram@38.122.20.226) Quit (Read error: Connection reset by peer)
[23:37] * alram (~alram@38.122.20.226) has joined #ceph
[23:40] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[23:41] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[23:42] * yanzheng (~zhyan@134.134.139.74) Quit (Remote host closed the connection)
[23:46] * onizo (~onizo@wsip-70-166-5-159.sd.sd.cox.net) Quit (Read error: Connection reset by peer)
[23:48] * rendar (~s@host23-191-dynamic.11-87-r.retail.telecomitalia.it) Quit ()
[23:48] * onizo (~onizo@wsip-70-166-5-159.sd.sd.cox.net) has joined #ceph
[23:50] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[23:51] * ScOut3R (~scout3r@4E5C7289.dsl.pool.telekom.hu) Quit ()
[23:53] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Quit: ...)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.