#ceph IRC Log

Index

IRC Log for 2014-01-02

Timestamps are in GMT/BST.

[0:02] * DarkAceZ (~BillyMays@50-32-23-165.drr01.hrbg.pa.frontiernet.net) Quit (Ping timeout: 480 seconds)
[0:06] * jnq (~jon@gruidae.jonquinn.com) Quit (Quit: WeeChat 0.3.7)
[0:08] * AfC (~andrew@101.119.14.19) has joined #ceph
[0:09] * AfC (~andrew@101.119.14.19) Quit ()
[0:23] * danieagle (~Daniel@186.214.63.138) Quit (Quit: inte+ e Obrigado Por tudo mesmo! :-D)
[0:24] * i_m (~ivan.miro@95.180.8.206) has joined #ceph
[0:39] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[0:40] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[0:59] * dmsimard2 (~Adium@108.163.152.2) Quit (Ping timeout: 480 seconds)
[1:00] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[1:01] * Cube1 (~Cube@12.248.40.138) has joined #ceph
[1:02] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[1:04] * DarkAceZ (~BillyMays@50-32-22-229.drr01.hrbg.pa.frontiernet.net) has joined #ceph
[1:07] * Cube (~Cube@12.248.40.138) Quit (Ping timeout: 480 seconds)
[1:12] * nhm (~nhm@71-13-153-2.static.ftbg.wi.charter.com) has joined #ceph
[1:12] * ChanServ sets mode +o nhm
[1:29] * mattbenjamin (~matt@76-206-42-105.lightspeed.livnmi.sbcglobal.net) has joined #ceph
[1:44] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) has joined #ceph
[1:44] * AfC (~andrew@215.114.154.202.sta.commander.net.au) has joined #ceph
[1:45] * mozg (~andrei@46.229.149.194) Quit (Ping timeout: 480 seconds)
[1:51] * mattbenjamin (~matt@76-206-42-105.lightspeed.livnmi.sbcglobal.net) Quit (Quit: Leaving.)
[1:55] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[2:02] * mschiff (~mschiff@port-2825.pppoe.wtnet.de) has joined #ceph
[2:04] * i_m (~ivan.miro@95.180.8.206) Quit (Ping timeout: 480 seconds)
[2:08] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[2:10] * mschiff_ (~mschiff@port-14414.pppoe.wtnet.de) Quit (Ping timeout: 480 seconds)
[2:12] * Discovery (~Discovery@192.162.100.197) has joined #ceph
[2:13] * yanzheng (~zhyan@134.134.139.76) has joined #ceph
[2:30] * AfC (~andrew@215.114.154.202.sta.commander.net.au) Quit (Quit: Leaving.)
[2:38] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[2:39] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) Quit (Quit: Leaving.)
[2:46] * lofejndif (~lsqavnbok@lumumba.torservers.net) has joined #ceph
[2:50] * Discovery (~Discovery@192.162.100.197) Quit (Read error: Connection reset by peer)
[2:50] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[2:51] * AfC (~andrew@215.114.154.202.sta.commander.net.au) has joined #ceph
[2:51] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[2:52] * diegows (~diegows@190.190.17.57) Quit (Ping timeout: 480 seconds)
[3:05] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[3:05] * AfC (~andrew@215.114.154.202.sta.commander.net.au) Quit (Quit: Leaving.)
[3:06] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[3:09] * ScOut3R (~scout3r@540205B4.dsl.pool.telekom.hu) Quit ()
[3:16] * flaxy (~afx@78.130.171.68) Quit (Quit: WeeChat 0.4.2)
[3:18] * flaxy (~afx@78.130.171.68) has joined #ceph
[3:22] * lofejndif (~lsqavnbok@8JQAAD60D.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)
[3:26] * shang (~ShangWu@220-135-203-169.HINET-IP.hinet.net) has joined #ceph
[3:26] * mschiff (~mschiff@port-2825.pppoe.wtnet.de) Quit (Remote host closed the connection)
[4:05] * yanzheng (~zhyan@134.134.139.76) Quit (Remote host closed the connection)
[4:07] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[4:20] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Quit: later)
[4:26] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[5:07] * Cube1 (~Cube@12.248.40.138) Quit (Read error: Operation timed out)
[5:28] * AfC (~andrew@101.119.14.19) has joined #ceph
[5:32] * shang (~ShangWu@220-135-203-169.HINET-IP.hinet.net) Quit (Read error: Operation timed out)
[5:32] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) has joined #ceph
[5:37] * dmsimard1 (~Adium@108.163.152.66) has joined #ceph
[5:39] * dmsimard (~Adium@69-165-206-93.cable.teksavvy.com) Quit (Read error: Operation timed out)
[5:50] * Vacum (~vovo@88.130.211.69) has joined #ceph
[5:51] * Sysadmin88 (~IceChat77@2.218.8.40) has joined #ceph
[5:56] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[5:57] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[5:57] * Vacum_ (~vovo@i59F4AF95.versanet.de) Quit (Ping timeout: 480 seconds)
[5:57] * AfC (~andrew@101.119.14.19) Quit (Quit: Leaving.)
[5:59] * nhm (~nhm@71-13-153-2.static.ftbg.wi.charter.com) Quit (Read error: Operation timed out)
[5:59] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[6:01] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[6:02] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[6:06] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit ()
[6:19] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[6:21] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[6:27] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[6:33] * bdonnahue (~tschneide@ool-18bda2d8.dyn.optonline.net) Quit (Quit: Leaving.)
[6:35] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[7:08] * shang (~ShangWu@111-248-172-113.dynamic.hinet.net) has joined #ceph
[7:08] * iaXe (~axe@223.223.202.195) has joined #ceph
[7:09] * shang_ (~ShangWu@111-248-172-113.dynamic.hinet.net) has joined #ceph
[7:09] * shang (~ShangWu@111-248-172-113.dynamic.hinet.net) Quit (Remote host closed the connection)
[7:09] * shang_ (~ShangWu@111-248-172-113.dynamic.hinet.net) Quit (Remote host closed the connection)
[7:11] * shang (~ShangWu@111-248-186-104.dynamic.hinet.net) has joined #ceph
[7:17] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[7:25] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[7:27] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[7:55] * mattt_ (~textual@92.52.76.140) has joined #ceph
[7:55] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[8:24] * Sysadmin88 (~IceChat77@2.218.8.40) Quit (Quit: ASCII a stupid question, get a stupid ANSI!)
[8:34] * wogri_risc (~wogri_ris@ro.risc.uni-linz.ac.at) has joined #ceph
[8:38] * peedu (~peedu@adsl89.uninet.ee) has joined #ceph
[8:51] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[8:57] * garphy`aw is now known as garphy
[9:04] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) has joined #ceph
[9:09] * rendar (~s@87.19.182.34) has joined #ceph
[9:19] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[9:22] * Muhlemmer (~kvirc@cable-90-50.zeelandnet.nl) has joined #ceph
[9:27] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[9:35] * ScOut3R (~scout3r@540205B4.dsl.pool.telekom.hu) has joined #ceph
[9:45] * KindTwo (KindOne@50.96.227.175) has joined #ceph
[9:46] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[9:46] * KindTwo is now known as KindOne
[10:06] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[10:07] * tziOm (~bjornar@194.19.106.242) has joined #ceph
[10:19] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[10:21] * shang (~ShangWu@111-248-186-104.dynamic.hinet.net) Quit (Ping timeout: 480 seconds)
[10:24] * hjjg (~hg@p3EE33666.dip0.t-ipconnect.de) has joined #ceph
[10:27] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[10:27] * cmdrk (~lincoln@c-24-12-206-91.hsd1.il.comcast.net) Quit (Quit: leaving)
[10:27] * i_m (~ivan.miro@95.180.8.206) has joined #ceph
[10:34] * houkouonchi-home (~linux@houkouonchi-1-pt.tunnel.tserv15.lax1.ipv6.he.net) Quit (Ping timeout: 480 seconds)
[11:02] <ccourtaut> morning
[11:14] * Cedric (~Cedric@185.23.92.11) has joined #ceph
[11:15] * Cedric is now known as cedrics
[11:19] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[11:27] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[11:32] * Cedric (~Cedric@185.23.92.11) has joined #ceph
[11:37] * cedrics (~Cedric@185.23.92.11) Quit (Ping timeout: 480 seconds)
[11:43] * haomaiwa_ (~haomaiwan@117.79.232.187) has joined #ceph
[11:47] * thorus (~jonas@82.199.158.66) Quit (Read error: Connection reset by peer)
[11:50] * haomaiwang (~haomaiwan@117.79.232.136) Quit (Ping timeout: 480 seconds)
[12:01] * KindOne (KindOne@0001a7db.user.oftc.net) Quit (Ping timeout: 480 seconds)
[12:10] * Cedric (~Cedric@185.23.92.11) Quit (Ping timeout: 480 seconds)
[12:11] * KindOne (KindOne@0001a7db.user.oftc.net) has joined #ceph
[12:17] * capri (~capri@212.218.127.222) has joined #ceph
[12:18] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[12:23] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Ping timeout: 480 seconds)
[12:29] * shang (~ShangWu@host-13.14-185-111.static.totalbb.net.tw) has joined #ceph
[12:32] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) Quit (Quit: Leaving.)
[12:38] * allsystemsarego (~allsystem@5-12-241-225.residential.rdsnet.ro) has joined #ceph
[12:54] * julian (~julianwa@125.70.133.91) Quit (Read error: Connection reset by peer)
[12:55] * julian (~julianwa@125.70.133.91) has joined #ceph
[12:55] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) has joined #ceph
[12:59] * jnq (~jon@95.85.22.50) has joined #ceph
[12:59] * mozg (~andrei@46.229.149.194) has joined #ceph
[13:01] * jnq (~jon@95.85.22.50) Quit ()
[13:01] * jnq (~jon@95.85.22.50) has joined #ceph
[13:14] * dmsimard1 (~Adium@108.163.152.66) Quit (Ping timeout: 480 seconds)
[13:15] * Cedric (~Cedric@185.23.92.11) has joined #ceph
[13:18] * shang (~ShangWu@host-13.14-185-111.static.totalbb.net.tw) Quit (Ping timeout: 480 seconds)
[13:20] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[13:22] * houkouonchi-home (~linux@66-215-209-207.dhcp.rvsd.ca.charter.com) has joined #ceph
[13:26] * shang (~ShangWu@111-241-236-224.dynamic.hinet.net) has joined #ceph
[13:28] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[13:29] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[13:30] * zidarsk8 (~zidar@84-255-203-33.static.t-2.net) has joined #ceph
[13:31] * zidarsk8 (~zidar@84-255-203-33.static.t-2.net) has left #ceph
[13:39] * mikedawson (~chatzilla@c-98-220-189-67.hsd1.in.comcast.net) Quit (Quit: ChatZilla 0.9.90.1 [Firefox 26.0/20131205075310])
[13:40] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[13:40] * ChanServ sets mode +v andreask
[13:53] * iaXe (~axe@223.223.202.195) Quit (Ping timeout: 480 seconds)
[13:54] * nerdtron (~oftc-webi@202.60.8.250) has joined #ceph
[13:54] <nerdtron> hi all
[13:55] <nerdtron> i would like to ask what happens if I have a pool with size 1 and min_size 1
[13:55] <nerdtron> and then one of my osd goes down, will all data be lost?
[13:56] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[13:56] * diegows (~diegows@190.190.17.57) has joined #ceph
[13:57] <wogri_risc> hi nerdtron
[13:57] <wogri_risc> if you can't recover the osd that has gone down, your data is lost.
[13:57] <wogri_risc> you will have to trash the pool and recreate it.
[13:57] <nerdtron> i don't care if data is lost as it is only a temp storage, will the pool stop functioning altogether?
[13:58] <wogri_risc> well, the pool is gone. as I said, you will have to trash the pool and recreate it, at least that's how I did it when I tried exactly your experiment.
[13:58] <nerdtron> ouch. so it means i have to at least set size to 2 or 3 for osd failure protection?
[13:59] <wogri_risc> hm. yes. but what's the problem with thrashing a pool and recreating it, if we're talking about temp storage?
[14:02] <nerdtron> I like to set it's replication to just 1 to speed up write speed, but since I still have to "intervene and recreate" means I still have to monitor it
[14:02] <nerdtron> i thought as long as other osd are available, the pool will still accept writes
[14:04] <wogri_risc> nope. think of it as a raid0 with a failed disk.
[14:04] <wogri_risc> I may be wrong though, I've tried this with RBD images.
[14:05] * Cedric (~Cedric@185.23.92.11) Quit (Read error: Operation timed out)
[14:05] * Cedric (~Cedric@185.23.92.11) has joined #ceph
[14:05] <nerdtron> oh right raid0 with failed disk
[14:05] <nerdtron> thanks for the info
[14:06] <darkfader> the rbd images are striped over all osds
[14:06] <wogri_risc> true.
[14:06] <darkfader> thats why it's so fatal
[14:06] <darkfader> or final
[14:06] <darkfader> :)
[14:06] <wogri_risc> :)
[14:07] <darkfader> wogri_risc: did you try adding a new osd to the murdered pool?
[14:07] <darkfader> it's interesting, sorry to nag you with that
[14:08] <wogri_risc> darkfader: i think I did, but I can't remember exactly.
[14:08] <darkfader> thanks
[14:08] <wogri_risc> it's easy to try on a test-system though.
[14:08] <darkfader> i'll write it down as something i wanna test
[14:13] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[14:14] * nerdtron (~oftc-webi@202.60.8.250) Quit (Quit: Page closed)
[14:20] * Cedric (~Cedric@185.23.92.11) Quit (Ping timeout: 480 seconds)
[14:20] * Cedric (~Cedric@185.23.92.11) has joined #ceph
[14:20] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[14:20] * mschiff (~mschiff@port-2825.pppoe.wtnet.de) has joined #ceph
[14:21] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Ping timeout: 480 seconds)
[14:23] * diegows (~diegows@190.190.17.57) Quit (Ping timeout: 480 seconds)
[14:24] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[14:25] * nerdtron (~oftc-webi@202.60.8.250) has joined #ceph
[14:25] <nerdtron> i need help again
[14:25] <nerdtron> one of my osd if flapping in up down state
[14:26] <nerdtron> my cluster is 0.003% degraded how do i start fixing this?
[14:26] <nerdtron> 26/726051 degraded (0.004%)
[14:27] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[14:27] <wogri_risc> find out why the osd is flapping.
[14:27] <wogri_risc> there will be sth in the logs
[14:28] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[14:28] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[14:29] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[14:29] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[14:30] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[14:31] <nerdtron> uhmm where are the log? i don't think i have /var/log/ceph
[14:33] * rudolfsteiner (~federicon@181.167.96.123) has joined #ceph
[14:37] * Muhlemmer (~kvirc@cable-90-50.zeelandnet.nl) Quit (Quit: KVIrc 4.3.1 Aria http://www.kvirc.net/)
[14:38] <wogri_risc> on my server (ubuntu, installed with ceph-deploy) it's /var/log/ceph/ceph-osd.0.log
[14:41] * shang (~ShangWu@111-241-236-224.dynamic.hinet.net) Quit (Quit: Ex-Chat)
[14:41] <nerdtron> i think i delete those long time ago..won't the server create new logs?
[14:42] <wogri_risc> are you complaining that you don't have logs because you have deleted them? :)
[14:48] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[14:51] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[14:54] * rudolfsteiner (~federicon@181.167.96.123) Quit (Quit: rudolfsteiner)
[14:55] <saaby_> hi guys
[14:56] <saaby_> should I be worried that my osd map epoch is now higher than 400.000 ?
[14:56] <saaby_> the cluster is almost a year old
[14:56] <saaby_> can anyone remember if the osdmap is 32bit or 64bit as the pg map?
[14:57] <saaby_> ...that is the osd map epoch..
[14:58] <aarontc> nerdtron: if you deleted the log files and haven't restarted the daemons since, the file may exist in a deleted state and be recoverable :)
[14:58] * iaXe (axe@123.116.43.193) has joined #ceph
[14:59] <wogri_risc> saaby_: you still have some time until you hit the 32bit limit of 4294967296
[15:00] <saaby_> yeah.. but still, we are planning to expand this cluster quite a bit..
[15:00] <wogri_risc> I think this is a question for #ceph-devel.
[15:01] <saaby_> right.. I'll go there.
[15:01] <saaby_> thanks
[15:08] * wogri_risc (~wogri_ris@ro.risc.uni-linz.ac.at) Quit (Remote host closed the connection)
[15:09] * rudolfsteiner (~federicon@181.167.96.123) has joined #ceph
[15:13] * pvsa (~pvsa@pd95c6a80.dip0.t-ipconnect.de) has joined #ceph
[15:19] * rudolfsteiner (~federicon@181.167.96.123) Quit (Quit: rudolfsteiner)
[15:21] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[15:23] * iaXe (axe@123.116.43.193) Quit (Ping timeout: 480 seconds)
[15:29] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[15:32] * pvsa (~pvsa@pd95c6a80.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[15:34] * scuttlemonkey (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) has joined #ceph
[15:34] * ChanServ sets mode +o scuttlemonkey
[15:37] * scuttlemonkey_ (~scuttlemo@c-69-244-181-5.hsd1.mi.comcast.net) Quit (Remote host closed the connection)
[15:40] * bdonnahue (~tschneide@ool-18bda2d8.dyn.optonline.net) has joined #ceph
[15:41] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[15:42] * peedu (~peedu@adsl89.uninet.ee) Quit (Remote host closed the connection)
[15:43] <bdonnahue> can anyone help with a ceph deploy issue. it looks like everything is installing correctly but for some reason the OSDs are not being added to the cluster
[15:44] * pvsa (~pvsa@89.204.139.9) has joined #ceph
[15:45] * hijacker (~hijacker@bgva.sonic.taxback.ess.ie) Quit (Quit: Leaving)
[15:45] * mobile (~pvsa@pd95c6a80.dip0.t-ipconnect.de) has joined #ceph
[15:45] * nhm (~nhm@71-13-153-2.static.ftbg.wi.charter.com) has joined #ceph
[15:45] * ChanServ sets mode +o nhm
[15:53] * pvsa (~pvsa@89.204.139.9) Quit (Ping timeout: 480 seconds)
[15:54] * mobile (~pvsa@pd95c6a80.dip0.t-ipconnect.de) Quit (Remote host closed the connection)
[16:02] * nhm (~nhm@71-13-153-2.static.ftbg.wi.charter.com) Quit (Quit: Lost terminal)
[16:03] * markbby (~Adium@168.94.245.4) has joined #ceph
[16:08] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[16:08] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[16:09] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[16:15] * diegows (~diegows@190.210.59.50) has joined #ceph
[16:15] * nerdtron (~oftc-webi@202.60.8.250) Quit (Quit: Page closed)
[16:21] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[16:29] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[16:30] * DarkAceZ (~BillyMays@50-32-22-229.drr01.hrbg.pa.frontiernet.net) Quit (Ping timeout: 480 seconds)
[16:30] * Discovery (~Discovery@192.162.100.197) has joined #ceph
[16:32] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[16:40] * rudolfsteiner (~federicon@200.68.116.185) has joined #ceph
[16:45] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) has joined #ceph
[16:45] * ChanServ sets mode +v andreask
[16:52] * peedu (~peedu@adsl89.uninet.ee) has joined #ceph
[16:59] <bdonnahue> i just setup an MDS and am trying to set it up for nfs use
[16:59] <bdonnahue> can anyone talk to me abotu this in high level
[17:00] * peedu (~peedu@adsl89.uninet.ee) Quit (Ping timeout: 480 seconds)
[17:02] * zerick (~eocrospom@190.187.21.53) has joined #ceph
[17:05] * vata (~vata@2607:fad8:4:6:8807:7f32:a45a:7c7e) has joined #ceph
[17:05] * Cedric (~Cedric@185.23.92.11) Quit (Quit: Quitte)
[17:06] * BillK (~BillK-OFT@106-69-25-13.dyn.iinet.net.au) Quit (Read error: Operation timed out)
[17:08] <aarontc> bdonnahue: I can try to help. What's your specific question/concern?
[17:09] <bdonnahue> so my first question is: in the documentation the mds is always installed on the same machine as the monitor
[17:09] <bdonnahue> why is this/is this required
[17:10] <aarontc> It is neither required nor recommended.. that is just a configuration suitable for testing ceph (I assume you saw this in the quick start guide)
[17:11] <aarontc> the MDS uses gobs more RAM than a monitor node will need :)
[17:12] <aarontc> the only requirements for MDS are that it have a "public" IP (if you're doing separate public and cluster networks) and that it be reachable by all clients wanting to mount CephFS. It also must be able to reach all OSDs
[17:12] <bdonnahue> ah ok. i have installed my mds on a seperate machine. my second question is:
[17:12] <bdonnahue> when the mds is added to the pool im asumign that it has created a / directory somewhere? or am i mistaken?
[17:13] <aarontc> by default the MDS uses the "metadata" pool to store all its metadata, including directory paths.. so you will have data in the "metadata" pool after the first MDS boot, yes
[17:14] * mattbenjamin (~matt@aa2.linuxbox.com) has joined #ceph
[17:15] <bdonnahue> ok so now I would like to mount the file system via nfs. Im wondering if its easy to set that up
[17:15] <aarontc> you will need a node which mounts CephFS somewhere (maybe /export/ceph"?) and then serves that out via nfsd
[17:15] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[17:19] <bdonnahue> ah ok so ill need a ddidcated nfs server. would it be possible to have multiple server running concurrently?
[17:20] <aarontc> Yes, but your NFS clients might get confused sometimes, since there is no mechanism in place for CephFS to notify nfsd that things have changed, AFAIK
[17:21] <bdonnahue> ah i see
[17:22] <bdonnahue> im reading throuhg the mount ceph fs page; it looks like that is the process by which the node running the nfsd should mount ceph?
[17:22] <aarontc> correct
[17:22] <bdonnahue> ok and then that node would simply export the mounted dir?
[17:22] <aarontc> correct
[17:23] <bdonnahue> ok. so the last thing that is confusing me right now is permissions. how can the nfsd pass through the authentication etc
[17:24] <aarontc> it doesn't. cephfs has all the normal POSIX permission semantics, so whatever you do on top of that should work fine
[17:25] <bdonnahue> ok thanks for the help. i am going to try to get this running later today
[17:25] * LeaChim (~LeaChim@host86-161-89-52.range86-161.btcentralplus.com) has joined #ceph
[17:25] <aarontc> good luck
[17:25] <bdonnahue> thank
[17:25] <bdonnahue> s
[17:25] <aarontc> (btw, is there some reason you're using NFS instead of mounting CephFS on all your clients?)
[17:25] * DarkAceZ (~BillyMays@50-32-21-177.drr01.hrbg.pa.frontiernet.net) has joined #ceph
[17:26] <aarontc> also, are you aware CephFS is considered experimental at this time?
[17:26] <bdonnahue> how do you mean?
[17:27] <aarontc> I mean is there some reason you want to proxy CephFS behind NFS instead of directly mounting CephFS on the clients that you would be mounting NFS on?
[17:27] * Tamil1 (~Adium@cpe-76-168-18-224.socal.res.rr.com) has joined #ceph
[17:27] <bdonnahue> oh i see
[17:28] <bdonnahue> well the main reason im setting up nfs is to mount ceph from an esxi box
[17:28] <bdonnahue> id like to store vms on there
[17:28] <aarontc> oh, I wouldn't do that
[17:28] <bdonnahue> ?
[17:28] <aarontc> RBD is much more suited to storing single large files like VM images
[17:28] <aarontc> and CephFS is experimental enough that you will probably encounter issues
[17:30] * ScOut3R (~scout3r@540205B4.dsl.pool.telekom.hu) Quit ()
[17:30] <bdonnahue> oh i see. so forgive me if im wrong but a block device will appear as a harddrive?
[17:30] <aarontc> correct
[17:31] <aarontc> I believe the current methodology for using RBD images with vmware is to export them via iSCSI
[17:31] <aarontc> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-May/001519.html
[17:32] <bdonnahue> can multiple clients access the same rbd at the same time?
[17:32] <aarontc> Ceph has no problem with that, but your clients might if they don't know how to share a block device properly (cache synchronization, etc.)
[17:41] * hjjg_ (~hg@p3EE309A3.dip0.t-ipconnect.de) has joined #ceph
[17:42] * ircolle (~Adium@2601:1:8380:2d9:b8f9:3861:a64d:6238) has joined #ceph
[17:43] * hjjg (~hg@p3EE33666.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[17:44] <bdonnahue> ok thanks. that sounds like a much better plan
[17:44] <bdonnahue> so you said ceph fs is extremely experimental. im not sure what that means with respect to my data
[17:45] <bdonnahue> i know the mds is handling the meta data associated with the file system
[17:45] <bdonnahue> so im wondering what issues might be encountered
[17:45] <aarontc> well, it means you might want to think hard about whether you want to store production data in CephFS.
[17:46] <aarontc> I am using it in production, and so are a lot of other people, but I've encountered some major issues myself
[17:46] <aarontc> so I consider it quasi-production.. if I lose all my CephFS data tomorrow, I'll be unhappy but it won't be the end of the world
[17:46] <bdonnahue> ok
[17:46] <bdonnahue> is that a realistic scenario though?
[17:46] <aarontc> RBD is considered production though, so you shouldn't have any issues with it
[17:47] * sagelap (~sage@cpe-23-242-158-79.socal.res.rr.com) Quit (Read error: Operation timed out)
[17:47] * diegows (~diegows@190.210.59.50) Quit (Ping timeout: 480 seconds)
[17:47] <aarontc> well, the Ceph project has officially marked CephFS experimental, so you are free to use it but only you can determine whether you can accept those kinds of risks or not :)
[17:48] <bdonnahue> hmm ill have to think about that. the alternative being mount the rbd and then format it like a normal file system?
[17:49] <aarontc> when your RBD is attached to a VM the VM can do anything it wishes just like a normal hard disk
[17:49] <aarontc> traditionally you'd put 1 or more partitions on it and then format those with a filesystem
[17:50] <bdonnahue> hmm im trying to think of what the best course of action is here. I wanted to allow a mysql cluster to keep their data stores on ceph
[17:50] * mattt_ (~textual@92.52.76.140) Quit (Read error: Connection reset by peer)
[17:50] <bdonnahue> im not sure how ceph manages rbd's. ill have to look into that
[17:51] <aarontc> are you trying to have one filesystem shared by all the MySQL instances, or is each instance going to have its own filesystem and then use MySQL-level replication?
[17:52] <bdonnahue> the latter
[17:52] <aarontc> (I'm personally unaware of a method for doing the former, but I'm not a MySQL expert in that sense)
[17:52] <aarontc> Okay, then using an RBD per MySQL VM seems like the way to go
[17:53] <bdonnahue> this is a dumb question but is there a way to keep track of the rdb's ive created? ie some kind of ls command
[17:53] <aarontc> "rbd ls"
[17:53] <bdonnahue> lol wow
[17:53] * sagelap (~sage@2600:1012:b022:baeb:f9da:5c15:c56a:d42e) has joined #ceph
[17:56] <aarontc> you can use "rbd ls -l" to get image max-sizes also
[17:56] <aarontc> and if you put RBD images in pools other than "rbd" you can specify -p <poolname> as well
[17:57] <bdonnahue> i saw this concept of pool earlier but have not yet found a definition for it. is it just a collection of objects?
[17:58] <aarontc> bdonnahue: You probably should study http://ceph.com/docs/master/architecture/
[17:58] <bdonnahue> thanks thats a good idea
[17:58] <dwm> aarontc: I recall CephFS only being labelled as experimental in multi-MDS configurations?
[17:59] <aarontc> dwm: not according to http://ceph.com/docs/master/cephfs/
[18:00] <dwm> That's possibly at odds with (the more recently written?) https://wiki.ceph.com/FAQs/Is_Ceph_Production-Quality%3F
[18:01] <aarontc> well, I can't speak for Ceph in an official capacity, but I'd assume the officially released documentation is more "correct" than the wiki, personally
[18:01] <dwm> ("production quality", of course, being a somewhat subjective measure.)
[18:02] * garphy is now known as garphy`aw
[18:03] <aarontc> I, personally, have experienced data-loss-causing events several times with CephFS :)
[18:03] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) Quit (Quit: Leaving.)
[18:05] <aarontc> (but don't let that deter you, the Ceph engineers helped graciously and expeditiously in all cases! :))
[18:06] * bjornar (~bjornar@ti0099a340-dhcp0395.bb.online.no) has joined #ceph
[18:07] * bandrus (~Adium@c-98-238-176-251.hsd1.ca.comcast.net) has joined #ceph
[18:08] * rudolfsteiner (~federicon@200.68.116.185) Quit (Quit: rudolfsteiner)
[18:23] * rudolfsteiner (~federicon@200.68.116.185) has joined #ceph
[18:23] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[18:28] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[18:34] * rudolfsteiner (~federicon@200.68.116.185) Quit (Quit: rudolfsteiner)
[18:35] * xevwork (~xevious@6cb32e01.cst.lightpath.net) has joined #ceph
[18:42] * diegows (~diegows@190.210.59.50) has joined #ceph
[18:44] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[18:47] * davidzlap (~Adium@ip68-5-239-214.oc.oc.cox.net) has joined #ceph
[18:48] * xmltok (~xmltok@cpe-76-90-130-65.socal.res.rr.com) has joined #ceph
[18:50] * Cube (~Cube@66-87-65-110.pools.spcsdns.net) has joined #ceph
[18:51] * xmltok_ (~xmltok@cpe-76-90-130-65.socal.res.rr.com) has joined #ceph
[18:51] * xmltok (~xmltok@cpe-76-90-130-65.socal.res.rr.com) Quit (Read error: Connection reset by peer)
[18:53] * rudolfsteiner (~federicon@200.68.116.185) has joined #ceph
[18:56] * aarya (~aarya@115.242.29.225) has joined #ceph
[18:56] * hjjg_ (~hg@p3EE309A3.dip0.t-ipconnect.de) Quit (Read error: Operation timed out)
[18:59] * andreask (~andreask@h081217067008.dyn.cm.kabsi.at) Quit (Quit: Leaving.)
[19:00] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) has joined #ceph
[19:01] <aarya> Hi everyone, I want to make a custimize debian iso, I tired simple-cdd, but it did not work in 64 bit, I am able to create live cd, but now I want to create a custom debian iso, Please suggest me any other tool
[19:02] * xarses (~andreww@12.164.168.115) has joined #ceph
[19:06] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) Quit (Read error: Connection reset by peer)
[19:09] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[19:10] * DarkAce-Z (~BillyMays@50-32-33-135.drr01.hrbg.pa.frontiernet.net) has joined #ceph
[19:11] * Pedras (~Adium@c-67-188-26-20.hsd1.ca.comcast.net) has joined #ceph
[19:14] * jdmason (~jon@134.134.137.75) Quit (Remote host closed the connection)
[19:14] * DarkAceZ (~BillyMays@50-32-21-177.drr01.hrbg.pa.frontiernet.net) Quit (Ping timeout: 480 seconds)
[19:16] <aarontc> aarya: perhaps ask in #debian?
[19:17] * dmsimard (~Adium@108.163.152.2) Quit (Ping timeout: 480 seconds)
[19:17] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[19:19] * jdmason (~jon@192.55.54.40) has joined #ceph
[19:22] * wusui (~Warren@2607:f298:a:607:95bf:9cca:f49b:c39b) has joined #ceph
[19:23] * Dark-Ace-Z (~BillyMays@50-32-46-251.drr01.hrbg.pa.frontiernet.net) has joined #ceph
[19:26] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[19:27] * DarkAce-Z (~BillyMays@50-32-33-135.drr01.hrbg.pa.frontiernet.net) Quit (Ping timeout: 480 seconds)
[19:33] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) Quit (Quit: Leaving.)
[19:35] * sarob (~sarob@nat-dip31-wl-e.cfw-a-gci.corp.yahoo.com) has joined #ceph
[19:36] * mikedawson (~chatzilla@23-25-46-97-static.hfc.comcastbusiness.net) has joined #ceph
[19:36] * sarob (~sarob@nat-dip31-wl-e.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[19:37] * sarob (~sarob@nat-dip31-wl-e.cfw-a-gci.corp.yahoo.com) has joined #ceph
[19:38] * sarob (~sarob@nat-dip31-wl-e.cfw-a-gci.corp.yahoo.com) Quit (Remote host closed the connection)
[19:39] * sarob (~sarob@2001:4998:effd:600:281c:d68a:419f:c15b) has joined #ceph
[19:48] * LPG (~LPG@c-76-104-197-224.hsd1.wa.comcast.net) has joined #ceph
[19:49] * bandrus1 (~Adium@107.222.158.227) has joined #ceph
[19:52] * dmick (~dmick@2607:f298:a:607:3db1:e6ae:1ece:5c0b) has joined #ceph
[19:56] * bandrus (~Adium@c-98-238-176-251.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[20:01] * aarya (~aarya@115.242.29.225) Quit (Remote host closed the connection)
[20:03] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) Quit (Quit: tryggvil)
[20:03] * zidarsk8 (~zidar@89-212-28-144.dynamic.t-2.net) has joined #ceph
[20:03] * sagelap1 (~sage@38.122.20.226) has joined #ceph
[20:03] * zidarsk8 (~zidar@89-212-28-144.dynamic.t-2.net) has left #ceph
[20:03] * alexp789 (~alexp789@host81-151-154-67.range81-151.btcentralplus.com) has joined #ceph
[20:08] * sagelap (~sage@2600:1012:b022:baeb:f9da:5c15:c56a:d42e) Quit (Ping timeout: 480 seconds)
[20:10] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) has joined #ceph
[20:16] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) Quit (Quit: tryggvil)
[20:23] <bdonnahue> i want to mount ceph rbd from esxi has anyone tried this?
[20:24] * ScOut3R (~scout3r@540205B4.dsl.pool.telekom.hu) has joined #ceph
[20:25] <dmick> I believe I've heard of users using the stgt iSCSI gateway for that
[20:30] <bdonnahue> yea im trying to found a scuess story and a guide to doing so
[20:32] <dmick> ah. yeah, dunno
[20:32] <dmick> fwiw once the gateway is set up the iscsi target looks pretty standard (i.e. no ceph tweaks should be required on the VMWare side)
[20:33] * sarob (~sarob@2001:4998:effd:600:281c:d68a:419f:c15b) Quit (Remote host closed the connection)
[20:33] <bdonnahue> im a bit unfamiliar with iSCSI. what are the advantages of doing that vs nfs mount
[20:33] * Sysadmin88 (~IceChat77@2.218.8.40) has joined #ceph
[20:34] * Sysadmin88 (~IceChat77@2.218.8.40) Quit ()
[20:34] * Sysadmin88 (~IceChat77@2.218.8.40) has joined #ceph
[20:35] <dmick> iscsi->rbd is a simpler path, and could be more performant (don't know if it is). Also it for sure uses the userland librbd, which tends to lead kernel rbd in features (if nfs would rely on krbd)
[20:35] <dmick> you definitely don't need any of the filesystem code
[20:35] <bdonnahue> im worried that if i sweitch from esxi to xenserver it might be easier for both to mount nfs?
[20:37] <dmick> don't know much about xenserver, but I'd be surprised if anything couldn't use iSCSI.
[20:38] <bdonnahue> i wonder if it would be able to mount the device when it was formatted my esxi though
[20:39] <dmick> that's probably independent of access method
[20:39] <bdonnahue> im reading into it. i think youre right... very new to iSCSI sorry that now seems like a very dumb question
[20:40] <dmick> lots of moving parts, no worries
[20:40] * IndraPr (~indra@bb121-6-34-250.singnet.com.sg) has joined #ceph
[20:40] * rudolfsteiner (~federicon@200.68.116.185) Quit (Quit: rudolfsteiner)
[20:40] <IndraPr> hi - anyone can help? one of my osds is full (reaching 95%) and i am having problems rebalancing it
[20:41] <IndraPr> i have put the workaround by setting ceph pg set_full_ratio 0.98
[20:41] <IndraPr> but now the osd is reaching 97% and counting :(
[20:42] <IndraPr> dev/sdd1 458140932 444337876 13803056 97% /var/lib/ceph/osd/ceph-15
[20:43] <IndraPr> ceph osd reweight-by-utilization doesn't help
[20:43] <IndraPr> no change: average_util: 0.804776, overload_util: 0.965731. overloaded osds: (none)
[20:43] <dmick> manually reweight less?
[20:46] <IndraPr> can advise how to do that?
[20:47] * sarob (~sarob@nat-dip4.cfw-a-gci.corp.yahoo.com) has joined #ceph
[20:49] <bdonnahue> http://www.racktopsystems.com/choosing-nfs-over-iscsi/
[20:53] <IndraPr> dmick - is it using this command? ceph osd reweight {osd-num} {weight}
[20:58] <IndraPr> have tried
[20:58] <IndraPr> $ ceph osd reweight 15 0.40
[20:58] <IndraPr> reweighted osd.15 to 0.4 (8262142)
[20:58] <IndraPr> dev/sdd1 458140932 444419848 13721084 98% /var/lib/ceph/osd/ceph-15
[20:58] <IndraPr> i don't see it going down :(
[21:00] * wer (~wer@206-248-239-142.unassigned.ntelos.net) Quit (Read error: Operation timed out)
[21:01] * rudolfsteiner (~federicon@200.68.116.185) has joined #ceph
[21:01] * rudolfsteiner (~federicon@200.68.116.185) Quit ()
[21:08] <bdonnahue> can anyone explain in high level how a client connects to ceph to mount a rbd
[21:12] <bdonnahue> im getting confused about how to block device is created etc
[21:12] <mikedawson> bdonnahue: Here is my understanding...rbd client checks ceph.conf to learn about monitors, then uses its keyring to contacts monitors to get maps, then the uses librados to talk directly to osds. librdb sits on top of rados
[21:13] <mikedawson> s/librdb/librbd/
[21:13] <kraken> mikedawson meant to say: bdonnahue: Here is my understanding...rbd client checks ceph.conf to learn about monitors, then uses its keyring to contacts monitors to get maps, then the uses librados to talk directly to osds. librbd sits on top of rados
[21:15] <bdonnahue> ok so this really has nothing to do with the mds then?
[21:15] <mikedawson> bdonnahue: no mds'es are required for rbd
[21:16] <mikedawson> I don't even install them
[21:20] <bdonnahue> ok so first i need to create a rbd then i can mount it? on what machine would i issue the commands to create the rbd?
[21:25] * zerick (~eocrospom@190.187.21.53) Quit (Quit: Saliendo)
[21:25] * zerick (~eocrospom@190.187.21.53) has joined #ceph
[21:31] <mikedawson> bdonnahue: yes, you need to create the images first (http://ceph.com/docs/next/man/8/rbd/#examples). Those commands can be run on any machine that has the proper keyring
[21:32] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) has joined #ceph
[21:35] <bdonnahue> im still confused by the notion of pools. why have more than one in a cluster?
[21:38] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[21:40] <xarses> bdonnahue: say one is SSD's another is 10k disk, another is spin down
[21:40] <xarses> bdonnahue: or to separate access, or separate replication factors
[21:44] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) has joined #ceph
[21:45] <bdonnahue> ah i dindt realize you could segregate pools based on the osd's
[21:45] <bdonnahue> im reading up on that now
[21:46] * tryggvil (~tryggvil@17-80-126-149.ftth.simafelagid.is) Quit (Quit: tryggvil)
[21:46] <IndraPr> anyone can help: - health HEALTH_WARN 14 pgs backfill_toofull; 14 pgs stuck unclean; recovery 20064/1614024 degraded (1.243%); 4 near full osd(s)
[21:46] <IndraPr> how to resolve the backfill_toofull and stuck unclean issue?
[21:47] <IndraPr> it happened after i re-weight one of the osds which was near full
[21:48] <bdonnahue> so when i create a pool / rbd i doe this from any osd or the admin node?
[21:48] <xarses> bdonnahue: any machine with the admin key
[21:48] <dmick> bdonnahue: any ceph command (save a very few) can happen from any host that has the cluster credentials.
[21:48] <dmick> it's a cluster, not a host.
[21:50] <bdonnahue> ok thanks
[21:50] <dmick> IndraPr: reweighting won't move data off the OSD
[21:50] <dmick> I don't think
[21:51] <dmick> how many OSDs do you have, and are they all the same size?
[21:51] * Tamil1 (~Adium@cpe-76-168-18-224.socal.res.rr.com) has left #ceph
[21:54] * doppelgrau (~doppelgra@pd956d116.dip0.t-ipconnect.de) has joined #ceph
[22:06] * wer (~wer@206-248-239-142.unassigned.ntelos.net) has joined #ceph
[22:08] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) Quit (Quit: Leaving.)
[22:08] * wer (~wer@206-248-239-142.unassigned.ntelos.net) Quit (Read error: Connection reset by peer)
[22:08] * wer (~wer@206-248-239-142.unassigned.ntelos.net) has joined #ceph
[22:19] * diegows (~diegows@190.210.59.50) Quit (Ping timeout: 480 seconds)
[22:26] * jesus (~jesus@emp048-51.eduroam.uu.se) Quit (Read error: Operation timed out)
[22:30] * Tamil1 (~Adium@cpe-76-168-18-224.socal.res.rr.com) has joined #ceph
[22:33] <IndraPr> dmick - i have 4 servers with 4 osds each, so total i have 16 osds
[22:33] * capri (~capri@212.218.127.222) Quit (Ping timeout: 480 seconds)
[22:33] <IndraPr> all are on the same size (500 GB) except the last four (480 GB), not much difference
[22:33] <dmick> so the other OSDs must be getting pretty full too?
[22:33] * xarses (~andreww@12.164.168.115) Quit (Read error: Connection reset by peer)
[22:34] <IndraPr> $ ceph osd tree
[22:34] * xarses (~andreww@12.164.168.115) has joined #ceph
[22:34] <IndraPr> # id weight type name up/down reweight
[22:34] <IndraPr> -1 7.24 root default
[22:34] <IndraPr> -2 1.84 host ceph-osd-02
[22:34] <IndraPr> 0 0.46 osd.0 up 1
[22:34] <IndraPr> 1 0.46 osd.1 up 1
[22:34] <IndraPr> 2 0.46 osd.2 up 1
[22:34] <IndraPr> 3 0.46 osd.3 up 1
[22:34] <IndraPr> -3 1.84 host ceph-osd-01
[22:34] <IndraPr> 4 0.46 osd.4 up 1
[22:34] <IndraPr> 5 0.46 osd.5 up 1
[22:34] <IndraPr> 6 0.46 osd.6 up 1
[22:34] <IndraPr> 7 0.46 osd.7 up 1
[22:34] <IndraPr> -4 1.84 host ceph-osd-03
[22:34] <IndraPr> 8 0.46 osd.8 up 1
[22:34] <IndraPr> 9 0.46 osd.9 up 1
[22:34] <IndraPr> 10 0.46 osd.10 up 1
[22:34] <IndraPr> 11 0.46 osd.11 up 1
[22:34] <IndraPr> -5 1.72 host ceph-osd-04
[22:34] <IndraPr> 12 0.43 osd.12 up 1
[22:34] <IndraPr> 13 0.43 osd.13 up 1
[22:34] <IndraPr> 14 0.43 osd.14 up 1
[22:34] <IndraPr> 15 0.43 osd.15 up 0.8
[22:34] <IndraPr> earlier osd.15 reaching 97%
[22:34] <IndraPr> after i changed the weight to 0.8, it reduces
[22:35] <IndraPr> dev/sda1 458140932 359473452 98667480 79% /var/lib/ceph/osd/ceph-12
[22:35] <IndraPr> dev/sdb1 458140932 403276964 54863968 89% /var/lib/ceph/osd/ceph-13
[22:35] <IndraPr> dev/sdc1 458140932 417400240 40740692 92% /var/lib/ceph/osd/ceph-14
[22:35] <IndraPr> dev/sdd1 458140932 405123772 53017160 89% /var/lib/ceph/osd/ceph-15
[22:35] <IndraPr> that's the df result on host ceph-osd-04
[22:35] <IndraPr> now ceph-14 is increasing
[22:36] <dmick> please don't put multiline comments directly in the channel; use fpaste.org or the like
[22:36] <IndraPr> oh i see, sorry
[22:36] <dmick> and, it's moving data off? cool.
[22:36] <IndraPr> yes it's moving data off
[22:36] <mozg> hello guys
[22:36] <IndraPr> but then i guess it causes the backfill_toofull
[22:37] <IndraPr> indra@hv-kvm-01:~$ ceph health
[22:37] <IndraPr> HEALTH_WARN 14 pgs backfill_toofull; 14 pgs stuck unclean; recovery 19850/1605721 degraded (1.236%); 4 near full osd(s)
[22:37] <dmick> but you must be getting pretty close to cluster full, as that's a pretty symmetric OSD config
[22:37] <dmick> yeah, 4 of them near full
[22:37] <mozg> could someone tell me if this setting a global or a client setting: filestore max sync interval
[22:37] <dmick> you're going to have to add capacity or move data off
[22:37] <dmick> mozg: the filestore is what the OSD sits on top of, so it's about the OSDs
[22:38] <IndraPr> dmick - but the rest of the OSDs (0-11) are quite ok
[22:38] <mozg> dmick, thanks
[22:38] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) has joined #ceph
[22:38] * jesus (~jesus@emp048-51.eduroam.uu.se) has joined #ceph
[22:38] <IndraPr> not too sure why the particular OSDs on osd-04 is going full faster
[22:38] <dmick> what pool(s) are you using, and how many PGs are in those pools?
[22:39] <IndraPr> 600 pgs
[22:39] * bdonnahue (~tschneide@ool-18bda2d8.dyn.optonline.net) has left #ceph
[22:44] * sagelap (~sage@2600:1012:b012:65c8:f9da:5c15:c56a:d42e) has joined #ceph
[22:45] * sagelap1 (~sage@38.122.20.226) Quit (Ping timeout: 480 seconds)
[22:46] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has joined #ceph
[22:47] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has left #ceph
[22:49] * allsystemsarego (~allsystem@5-12-241-225.residential.rdsnet.ro) Quit (Quit: Leaving)
[22:49] * jerrad (~jerrad@pat-global.macpractice.net) Quit (Remote host closed the connection)
[22:50] * Dark-Ace-Z is now known as DarkAceZ
[22:53] * sroy (~sroy@modemcable105.167-21-96.mc.videotron.ca) has joined #ceph
[22:56] * markbby (~Adium@168.94.245.4) Quit (Quit: Leaving.)
[22:59] * Underbyte (~jerrad@pat-global.macpractice.net) has joined #ceph
[23:04] * sroy (~sroy@modemcable105.167-21-96.mc.videotron.ca) Quit (Quit: Quitte)
[23:06] * sagelap (~sage@2600:1012:b012:65c8:f9da:5c15:c56a:d42e) Quit (Ping timeout: 480 seconds)
[23:07] * angdraug (~angdraug@12.164.168.115) has joined #ceph
[23:10] * xevwork (~xevious@6cb32e01.cst.lightpath.net) Quit (Remote host closed the connection)
[23:11] * sagelap (~sage@cpe-23-242-158-79.socal.res.rr.com) has joined #ceph
[23:14] * mattt_ (~textual@cpc25-rdng20-2-0-cust162.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[23:18] * mkoderer (uid11949@id-11949.ealing.irccloud.com) Quit (Read error: Connection reset by peer)
[23:18] * mkoderer (uid11949@ealing.irccloud.com) has joined #ceph
[23:19] * sagelap (~sage@cpe-23-242-158-79.socal.res.rr.com) Quit (Ping timeout: 480 seconds)
[23:25] * rendar (~s@87.19.182.34) Quit ()
[23:28] * BillK (~BillK-OFT@106-69-25-13.dyn.iinet.net.au) has joined #ceph
[23:31] * seif (sid11725@id-11725.ealing.irccloud.com) Quit (Read error: Connection reset by peer)
[23:31] * seif (sid11725@id-11725.ealing.irccloud.com) has joined #ceph
[23:34] * bjornar (~bjornar@ti0099a340-dhcp0395.bb.online.no) Quit (Ping timeout: 480 seconds)
[23:36] <mozg> guys, could someone run ceph osd perf and tell me what values they have in the second column.
[23:36] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) Quit (Quit: Leaving.)
[23:36] <mozg> i am getting between 150-250 ms range
[23:36] <mozg> which sounds way too much
[23:37] * vata (~vata@2607:fad8:4:6:8807:7f32:a45a:7c7e) Quit (Quit: Leaving.)
[23:40] * seif (sid11725@id-11725.ealing.irccloud.com) Quit (Ping timeout: 480 seconds)
[23:41] * seif (sid11725@2604:8300:100:200b:6667:2:0:2dcd) has joined #ceph
[23:51] * iaXe (axe@123.116.43.193) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.