#ceph IRC Log

Index

IRC Log for 2016-07-04

Timestamps are in GMT/BST.

[0:16] * freezza (~ladmin@185.13.112.77) Quit (Remote host closed the connection)
[0:57] * rburkholder (~overonthe@199.68.193.62) has joined #ceph
[1:00] * bjornar_ (~bjornar@ti0099a430-0410.bb.online.no) Quit (Ping timeout: 480 seconds)
[1:03] * dynamicudpate (~overonthe@199.68.193.62) Quit (Ping timeout: 480 seconds)
[1:06] * Altitudes (~Corneliou@tor2r.ins.tor.net.eu.org) has joined #ceph
[1:25] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[1:27] * calvinx (~calvin@101.100.172.246) Quit (Quit: calvinx)
[1:35] * Altitudes (~Corneliou@06SAAETG7.tor-irc.dnsbl.oftc.net) Quit ()
[1:49] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) Quit (Ping timeout: 480 seconds)
[1:55] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[1:55] * IvanJobs (~ivanjobs@183.192.78.179) Quit ()
[1:55] * hoonetorg (~hoonetorg@77.119.226.254.static.drei.at) has joined #ceph
[1:59] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:9901:29b4:b6c5:3fa9) has joined #ceph
[2:06] * borei (~dan@node-1w7jr9qle4x5ix2kjybp8d4fv.ipv6.telus.net) Quit (Ping timeout: 480 seconds)
[2:18] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:9901:29b4:b6c5:3fa9) Quit (Ping timeout: 480 seconds)
[2:25] * Unforgiven (~PappI@46.166.190.182) has joined #ceph
[2:47] * kevinc (~kevinc__@ip174-65-68-173.sd.sd.cox.net) has joined #ceph
[3:02] * Unforgiven (~PappI@4MJAAHDHO.tor-irc.dnsbl.oftc.net) Quit (Ping timeout: 480 seconds)
[3:09] * Solvius (~zviratko@46.166.190.206) has joined #ceph
[3:22] * kevinc (~kevinc__@ip174-65-68-173.sd.sd.cox.net) Quit (Quit: This computer has gone to sleep)
[3:36] * aarontc (~aarontc@2001:470:e893::1:1) has joined #ceph
[3:38] * Solvius (~zviratko@46.166.190.206) Quit ()
[3:39] * kevinc (~kevinc__@ip174-65-68-173.sd.sd.cox.net) has joined #ceph
[3:40] * sebastian-w (~quassel@212.218.8.139) Quit (Remote host closed the connection)
[3:40] * sebastian-w (~quassel@212.218.8.139) has joined #ceph
[3:44] * Jeffrey4l__ (~Jeffrey@119.251.238.8) Quit (Ping timeout: 480 seconds)
[3:46] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) has joined #ceph
[3:58] * dnunez (~dnunez@c-73-38-0-185.hsd1.ma.comcast.net) has joined #ceph
[4:03] * Jeffrey4l (~Jeffrey@121.16.246.42) has joined #ceph
[4:13] * kevinc (~kevinc__@ip174-65-68-173.sd.sd.cox.net) Quit (Quit: This computer has gone to sleep)
[4:19] * Eric1 (~raindog@tor2r.ins.tor.net.eu.org) has joined #ceph
[4:20] * allen_gao (~allen_gao@58.213.72.214) Quit (Quit: ZNC 1.6.2 - http://znc.in)
[4:22] * allen_gao (~allen_gao@58.213.72.214) has joined #ceph
[4:30] * yanzheng (~zhyan@125.70.20.240) has joined #ceph
[4:38] * flisky (~Thunderbi@210.12.157.94) has joined #ceph
[4:41] * borei (~dan@node-1w7jr9qle4x5ix2kjybp8d4fv.ipv6.telus.net) has joined #ceph
[4:43] * skinnejo (~skinnejo@173-27-199-104.client.mchsi.com) Quit (Quit: Leaving...)
[4:49] * Eric1 (~raindog@06SAAETK6.tor-irc.dnsbl.oftc.net) Quit ()
[4:51] * sankarshan (~sankarsha@121.244.87.117) has joined #ceph
[4:58] * Nats_ (~natscogs@114.31.195.238) has joined #ceph
[4:59] * Nats_ (~natscogs@114.31.195.238) Quit (Read error: Connection reset by peer)
[5:00] * Nats_ (~natscogs@114.31.195.238) has joined #ceph
[5:03] * Nats (~natscogs@114.31.195.238) Quit (Ping timeout: 480 seconds)
[5:09] * ChengPeng (~chris@180.168.126.179) has left #ceph
[5:09] * ChengPeng (~chris@180.168.126.179) has joined #ceph
[5:24] * Nats (~natscogs@114.31.195.238) has joined #ceph
[5:26] * alexxy (~alexxy@biod.pnpi.spb.ru) Quit (Quit: No Ping reply in 180 seconds.)
[5:27] * alexxy (~alexxy@biod.pnpi.spb.ru) has joined #ceph
[5:28] * sankarshan (~sankarsha@121.244.87.117) Quit (Quit: Are you sure you want to quit this channel (Cancel/Ok) ?)
[5:30] * Nats_ (~natscogs@114.31.195.238) Quit (Ping timeout: 480 seconds)
[5:31] * c0dice (~toodles@75-128-34-237.static.mtpk.ca.charter.com) has joined #ceph
[5:33] * codice (~toodles@75-128-34-237.static.mtpk.ca.charter.com) Quit (Ping timeout: 480 seconds)
[5:40] * Nats_ (~natscogs@114.31.195.238) has joined #ceph
[5:40] * IvanJobs (~ivanjobs@103.50.11.146) has joined #ceph
[5:40] <IvanJobs> hi cephers
[5:41] <IvanJobs> I'm testing ceph cluster, and don't known how to make rbd_cache config takes effects.
[5:42] <IvanJobs> I ceph-deploy push new ceph.conf to all nodes with new [client] section config, restart all osds, but when I use "ceph daemon osd.x config show" to check rbd cache config, they remains unchange.
[5:42] * dnunez (~dnunez@c-73-38-0-185.hsd1.ma.comcast.net) Quit (Remote host closed the connection)
[5:42] <IvanJobs> I don't know why, maybe my operations are not right. Thx in advance.
[5:42] * Vacuum__ (~Vacuum@88.130.193.2) has joined #ceph
[5:44] * derjohn_mob (~aj@x590c03e1.dyn.telefonica.de) has joined #ceph
[5:45] * Nats (~natscogs@114.31.195.238) Quit (Ping timeout: 480 seconds)
[5:49] * Vacuum_ (~Vacuum@i59F7942F.versanet.de) Quit (Ping timeout: 480 seconds)
[5:50] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[5:52] <Nats_> OSD's are not clients, so i would guess they dont actually parse that section of the config
[5:53] <IvanJobs> Nats_, actually I use fio with rbd engine to test ceph cluster.
[5:54] <IvanJobs> so I just modify ceph.conf, and restart my fio test. does that take effects?
[5:56] <IvanJobs> It is weried, although rbd_cache is client-side config, I can use "ceph daemon" command to check them. Make me confused.
[5:56] <Nats_> without being familiar with that tool, i would say yes
[5:56] <ChengPeng> it impact client not osd
[5:57] * gauravbafna (~gauravbaf@122.172.196.208) has joined #ceph
[5:57] * [0x4A6F]_ (~ident@p4FC2601A.dip0.t-ipconnect.de) has joined #ceph
[5:57] <Nats_> 'config show' dumps the value of all the tunable variables in "ceph" that are set within that OSD, even those that are completely irrelevant
[5:59] * [0x4A6F] (~ident@0x4a6f.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:59] * [0x4A6F]_ is now known as [0x4A6F]
[6:00] * derjohn_mobi (~aj@x590e6942.dyn.telefonica.de) has joined #ceph
[6:04] * derjohn_mob (~aj@x590c03e1.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[6:18] * kuku (~kuku@119.93.91.136) has joined #ceph
[6:20] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[6:22] <kuku> hello, good morning. Have anyone encountered this before?
[6:23] <kuku> http://dpaste.com/030KPDV
[6:23] <kuku> I can't run any ceph command
[6:26] * alexxy[home] (~alexxy@biod.pnpi.spb.ru) has joined #ceph
[6:30] * alexxy (~alexxy@biod.pnpi.spb.ru) Quit (Ping timeout: 480 seconds)
[6:34] * vikhyat (~vumrao@49.248.193.82) has joined #ceph
[6:38] * derjohn_mobi (~aj@x590e6942.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[6:43] * kuku (~kuku@119.93.91.136) Quit (Read error: Connection reset by peer)
[6:45] * rotbeard (~redbeard@185.32.80.238) has joined #ceph
[6:54] * kuku (~kuku@119.93.91.136) has joined #ceph
[6:55] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[7:00] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[7:02] * swami1 (~swami@49.44.57.239) has joined #ceph
[7:04] * TomasCZ (~TomasCZ@yes.tenlab.net) Quit (Quit: Leaving)
[7:04] * alexxy[home] (~alexxy@biod.pnpi.spb.ru) Quit (Quit: No Ping reply in 180 seconds.)
[7:05] * alexxy (~alexxy@biod.pnpi.spb.ru) has joined #ceph
[7:07] * nils_ (~nils_@doomstreet.collins.kg) has joined #ceph
[7:09] * gauravbafna (~gauravbaf@122.172.196.208) Quit (Remote host closed the connection)
[7:10] * nils_ (~nils_@doomstreet.collins.kg) Quit ()
[7:11] * allen_gao (~allen_gao@58.213.72.214) Quit (Remote host closed the connection)
[7:14] * vicente_ (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[7:15] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) Quit (Read error: Connection reset by peer)
[7:22] * swami1 (~swami@49.44.57.239) Quit (Ping timeout: 480 seconds)
[7:22] * yanzheng1 (~zhyan@125.70.20.240) has joined #ceph
[7:24] * yanzheng (~zhyan@125.70.20.240) Quit (Ping timeout: 480 seconds)
[7:32] * karnan (~karnan@121.244.87.117) has joined #ceph
[7:43] * derjohn_mobi (~aj@88.128.80.75) has joined #ceph
[7:44] * yanzheng (~zhyan@125.70.20.240) has joined #ceph
[7:45] * yanzheng1 (~zhyan@125.70.20.240) Quit (Ping timeout: 480 seconds)
[7:50] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[7:52] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) has joined #ceph
[8:00] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[8:18] * asalor (~asalor@0001ef37.user.oftc.net) Quit (Quit: leaving)
[8:19] * vicente_ (~~vicente@125-227-238-55.HINET-IP.hinet.net) has left #ceph
[8:19] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[8:19] * saintpablo (~saintpabl@gw01.mhitp.dk) has joined #ceph
[8:19] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) has joined #ceph
[8:19] * Meths (~meths@95.151.244.156) has joined #ceph
[8:20] * asalor (~asalor@218.83.broadband9.iol.cz) has joined #ceph
[8:22] * asalor (~asalor@218.83.broadband9.iol.cz) Quit ()
[8:22] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[8:23] * asalor (~asalor@218.83.broadband9.iol.cz) has joined #ceph
[8:24] * Meths (~meths@95.151.244.156) has joined #ceph
[8:25] * ade (~abradshaw@dslb-088-075-188-119.088.075.pools.vodafone-ip.de) has joined #ceph
[8:26] * rdas (~rdas@121.244.87.116) has joined #ceph
[8:26] * swami1 (~swami@49.38.0.153) has joined #ceph
[8:28] * pho0l (~pho0l@dsl-197-245-140-40.voxdsl.co.za) Quit (Read error: No route to host)
[8:29] * pho0l (~pho0l@dsl-197-245-140-40.voxdsl.co.za) has joined #ceph
[8:31] * gauravbafna (~gauravbaf@49.38.1.125) has joined #ceph
[8:37] * pho0l (~pho0l@dsl-197-245-140-40.voxdsl.co.za) Quit (Read error: Connection reset by peer)
[8:37] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[8:38] * kawa2014 (~kawa@89.184.114.246) has joined #ceph
[8:40] * derjohn_mobi (~aj@88.128.80.75) Quit (Ping timeout: 480 seconds)
[8:41] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[8:42] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) Quit ()
[8:42] * Meths (~meths@95.151.244.156) has joined #ceph
[8:42] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[8:53] * onyb (~ani07nov@27.34.248.81) has joined #ceph
[9:01] * ffilz (~ffilz@c-76-115-190-27.hsd1.or.comcast.net) Quit (Read error: Connection reset by peer)
[9:01] * ffilzwin (~ffilz@c-76-115-190-27.hsd1.or.comcast.net) Quit (Read error: Connection reset by peer)
[9:03] * karnan (~karnan@121.244.87.117) Quit (Ping timeout: 480 seconds)
[9:05] * ffilz (~ffilz@c-76-115-190-27.hsd1.or.comcast.net) has joined #ceph
[9:07] * branto (~branto@ip-78-102-208-181.net.upcbroadband.cz) has joined #ceph
[9:10] * karnan (~karnan@121.244.87.117) has joined #ceph
[9:10] * flisky (~Thunderbi@210.12.157.94) Quit (Quit: flisky)
[9:12] * kevinc (~kevinc__@ip174-65-68-173.sd.sd.cox.net) has joined #ceph
[9:12] * penguinRaider (~KiKo@69.163.33.182) Quit (Ping timeout: 480 seconds)
[9:13] * shyu (~Frank@218.241.172.114) has joined #ceph
[9:14] * theTrav (~theTrav@CPE-124-188-218-238.sfcz1.cht.bigpond.net.au) Quit (Remote host closed the connection)
[9:14] * kuku (~kuku@119.93.91.136) Quit (Read error: Connection reset by peer)
[9:19] * derjohn_mobi (~aj@fw.gkh-setu.de) has joined #ceph
[9:19] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[9:21] * evelu (~erwan@46.231.131.178) has joined #ceph
[9:22] * penguinRaider (~KiKo@14.139.82.6) has joined #ceph
[9:24] * kuku (~kuku@119.93.91.136) has joined #ceph
[9:32] * dgurtner (~dgurtner@84-73-130-19.dclient.hispeed.ch) has joined #ceph
[9:32] * theTrav (~theTrav@CPE-124-188-218-238.sfcz1.cht.bigpond.net.au) has joined #ceph
[9:37] * rendar (~I@host47-183-dynamic.7-87-r.retail.telecomitalia.it) has joined #ceph
[9:41] * Hemanth (~hkumar_@121.244.87.117) has joined #ceph
[9:46] * kevinc (~kevinc__@ip174-65-68-173.sd.sd.cox.net) Quit (Quit: This computer has gone to sleep)
[9:53] * nils_ (~nils_@doomstreet.collins.kg) has joined #ceph
[9:54] * MrBy (~jonas@85.115.23.2) has joined #ceph
[9:54] <mistur> hello
[9:55] <mistur> kuku, check if your MONs are runing
[9:58] * DanFoster (~Daniel@2a00:1ee0:3:1337:5a:d8fa:bd5f:acb8) Quit (Quit: Leaving)
[10:00] * dugravot6 (~dugravot6@l-p-dn-in-4a.lionnois.site.univ-lorraine.fr) Quit (Quit: Leaving.)
[10:01] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[10:02] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) has joined #ceph
[10:02] * allaok (~allaok@machine107.orange-labs.com) has joined #ceph
[10:03] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[10:04] * kuku (~kuku@119.93.91.136) Quit (Remote host closed the connection)
[10:04] * DanFoster (~Daniel@office.34sp.com) has joined #ceph
[10:05] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) has joined #ceph
[10:10] * TMM (~hp@178-84-46-106.dynamic.upc.nl) Quit (Ping timeout: 480 seconds)
[10:10] * onyb (~ani07nov@27.34.248.81) Quit (Ping timeout: 480 seconds)
[10:15] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) Quit (Remote host closed the connection)
[10:15] * rdias (~rdias@2001:8a0:749a:d01:f525:c32e:c1a4:34a0) has joined #ceph
[10:16] * darkfader (~floh@88.79.251.60) Quit (Remote host closed the connection)
[10:17] * darkfader (~floh@88.79.251.60) has joined #ceph
[10:19] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[10:24] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[10:27] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:30] * itamarl is now known as Guest1763
[10:30] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:32] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) has joined #ceph
[10:32] <nils_> so now I'm using jewel, and to still support the kernel driver I have set rbd default features = 1, however this does not enable striping
[10:35] * daviddcc (~dcasier@84.197.151.77.rev.sfr.net) Quit (Ping timeout: 480 seconds)
[10:35] * Guest1763 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[10:36] * itamarl is now known as Guest1765
[10:36] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:39] * Guest1765 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[10:42] * itamarl is now known as Guest1766
[10:42] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:42] <nils_> [14278434.884037] libceph: mon0 172.16.141.21:6789 feature set mismatch, my 102b84a842a42 < server's 40102b84a842a42, missing 400000000000000
[10:43] * billwebb (~billwebb@66.56.15.14) has joined #ceph
[10:43] * billwebb (~billwebb@66.56.15.14) Quit ()
[10:44] * billwebb (~billwebb@66.56.15.14) has joined #ceph
[10:45] * TMM (~hp@185.5.121.201) has joined #ceph
[10:46] * Guest1766 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[10:47] * itamarl is now known as Guest1767
[10:47] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:47] <nils_> I don't know what this id is, it may be tunables5?
[10:48] <BranchPredictor> monitor and client versions are incompatible
[10:48] <nils_> that's not it.
[10:48] <stein> either upgrade klient or unsethaspspool
[10:49] <stein> unset haspspool :)
[10:50] * Guest1767 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[10:50] <nils_> I think haspspool has a different id
[10:51] <stein> you are right, nevermind, i didt read it correctly
[10:51] <nils_> very likely chooseleaf_stable
[10:52] * derjohn_mobi (~aj@fw.gkh-setu.de) Quit (Ping timeout: 480 seconds)
[10:52] * billwebb (~billwebb@66.56.15.14) Quit (Ping timeout: 480 seconds)
[10:52] * itamarl is now known as Guest1770
[10:52] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:52] <nils_> live and learn...
[10:53] <nils_> so it is in fact the kernel version
[10:54] * Corti^carte (~Sliker@tor2r.ins.tor.net.eu.org) has joined #ceph
[10:55] * Guest1770 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[10:56] * itamarl is now known as Guest1771
[10:56] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[10:57] <The_Ball> can I identify the OSD involved from this output? 2016-07-04 10:56:40.082318 osd.5 [WRN] slow request 30.710215 seconds old, received at 2016-07-04 10:56:09.371961: osd_op(client.3630000.1:6178694 6.a8e5d15f rbd_data.190c7f2ae8944a.00000000000a1b2a [set-alloc-hint object_size 4194304 write_size 4194304,write 700416~1048576] snapc 0=[] ondisk+write e7642) currently waiting for subops from 7
[10:57] <The_Ball> Sorry
[10:57] <The_Ball> osd.5
[10:58] * takarider (~takarider@nat-pool-nrt-u1.redhat.com) has joined #ceph
[11:00] * derjohn_mobi (~aj@fw.gkh-setu.de) has joined #ceph
[11:00] * Guest1771 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:01] * MrBy (~jonas@85.115.23.2) Quit (Quit: This computer has gone to sleep)
[11:02] * MrBy (~jonas@85.115.23.2) has joined #ceph
[11:03] * ffilzwin (~ffilz@c-76-115-190-27.hsd1.or.comcast.net) has joined #ceph
[11:05] * MrBy (~jonas@85.115.23.2) Quit ()
[11:06] * itamarl is now known as Guest1772
[11:06] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:07] * MrBy (~jonas@85.115.23.2) has joined #ceph
[11:07] * MrBy (~jonas@85.115.23.2) Quit ()
[11:08] * Guest1772 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:11] * itamarl is now known as Guest1775
[11:11] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:13] * flisky (~Thunderbi@210.12.157.94) has joined #ceph
[11:13] * onyb (~ani07nov@27.34.248.81) has joined #ceph
[11:14] * Guest1775 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:16] * takarider (~takarider@nat-pool-nrt-u1.redhat.com) Quit (Quit: Textual IRC Client: www.textualapp.com)
[11:17] * itamarl is now known as Guest1776
[11:17] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:19] * Guest1776 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:22] * itamarl is now known as Guest1778
[11:22] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:23] * Corti^carte (~Sliker@7V7AAG1GF.tor-irc.dnsbl.oftc.net) Quit ()
[11:24] <badone> The_Ball: actually, 5 is waiting for 7
[11:25] * Guest1778 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:25] <badone> you should be able to search for rbd_data.190c7f2ae8944a.00000000000a1b2a in the logs for osd.7
[11:26] <badone> and you could try dump_historic_ops and dump_ops_in_flight too
[11:27] * itamarl is now known as Guest1779
[11:27] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:27] <The_Ball> badone, I used hdparm to turn off APM and acoustic management on all OSDs, seems to have removed the problem
[11:28] <badone> The_Ball: cool
[11:28] <The_Ball> home / shoestring budget cluster
[11:30] * Guest1779 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:32] * onyb (~ani07nov@27.34.248.81) Quit (Quit: raise SystemExit())
[11:32] * theTrav (~theTrav@CPE-124-188-218-238.sfcz1.cht.bigpond.net.au) Quit (Remote host closed the connection)
[11:35] * itamarl is now known as Guest1780
[11:35] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:37] * nastidon (~Frostshif@tsn109-201-152-5.dyn.nltelcom.net) has joined #ceph
[11:38] * Guest1780 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:42] * linjan (~linjan@176.195.217.168) has joined #ceph
[11:44] * shylesh (~shylesh@121.244.87.118) has joined #ceph
[11:45] * itamarl is now known as Guest1781
[11:45] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:48] * Guest1781 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:48] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:9901:29b4:b6c5:3fa9) has joined #ceph
[11:51] * itamarl is now known as Guest1782
[11:51] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[11:54] * theTrav (~theTrav@CPE-124-188-218-238.sfcz1.cht.bigpond.net.au) has joined #ceph
[11:55] * Guest1782 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[11:56] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[11:56] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) has joined #ceph
[11:57] * malevolent (~quassel@124.red-88-11-251.dynamicip.rima-tde.net) Quit (Ping timeout: 480 seconds)
[11:58] * itamarl is now known as Guest1783
[11:58] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:01] * Guest1783 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:02] * malevolent (~quassel@124.red-88-11-251.dynamicip.rima-tde.net) has joined #ceph
[12:04] * EinstCrazy (~EinstCraz@60-249-152-164.HINET-IP.hinet.net) Quit (Remote host closed the connection)
[12:07] * nastidon (~Frostshif@tsn109-201-152-5.dyn.nltelcom.net) Quit ()
[12:08] * itamarl is now known as Guest1784
[12:08] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:08] * Guest1784 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:13] * itamarl is now known as Guest1786
[12:13] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:15] * IvanJobs (~ivanjobs@103.50.11.146) Quit ()
[12:18] * theTrav (~theTrav@CPE-124-188-218-238.sfcz1.cht.bigpond.net.au) Quit (Remote host closed the connection)
[12:18] * Guest1786 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:19] * itamarl is now known as Guest1787
[12:19] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:21] * Guest1787 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:22] * itamarl is now known as Guest1788
[12:22] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:27] * Guest1788 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:27] * itamarl is now known as Guest1789
[12:27] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:31] * itamarl is now known as Guest1790
[12:31] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:32] * Guest1789 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:35] * Guest1790 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:36] * itamarl is now known as Guest1791
[12:36] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:39] * Guest1791 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:40] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[12:41] * tserong (~tserong@203-214-92-220.dyn.iinet.net.au) Quit (Quit: Leaving)
[12:41] * DV_ (~veillard@2001:41d0:a:f29f::1) Quit (Quit: Leaving)
[12:41] * DV_ (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[12:43] * itamarl is now known as Guest1792
[12:43] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[12:43] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:44] * rraja (~rraja@121.244.87.117) has joined #ceph
[12:46] * Guest1792 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:47] * kuku (~kuku@112.203.51.69) has joined #ceph
[12:49] * Meths (~meths@95.151.244.156) has joined #ceph
[12:49] * kuku (~kuku@112.203.51.69) Quit (Read error: Connection reset by peer)
[12:49] * kuku (~kuku@112.203.51.69) has joined #ceph
[12:50] * itamarl is now known as Guest1793
[12:50] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[12:53] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[12:53] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[12:53] * ngoswami (~ngoswami@121.244.87.116) Quit ()
[12:53] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[12:54] * Guest1793 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[12:57] * itamarl is now known as Guest1796
[12:57] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:00] * Guest1796 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:04] * shylesh (~shylesh@121.244.87.118) Quit (Remote host closed the connection)
[13:06] * vicente (~~vicente@125-227-238-55.HINET-IP.hinet.net) Quit (Quit: Leaving)
[13:07] * itamarl is now known as Guest1797
[13:07] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:10] * Guest1797 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:11] * swami1 (~swami@49.38.0.153) Quit (Read error: Connection reset by peer)
[13:13] * itamarl is now known as Guest1799
[13:13] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:17] * Guest1799 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:20] * itamarl is now known as Guest1800
[13:20] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:22] * penguinRaider (~KiKo@14.139.82.6) Quit (Ping timeout: 480 seconds)
[13:24] * Guest1800 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:26] * itamarl is now known as Guest1801
[13:26] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:28] * Guest1801 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:29] * itamarl is now known as Guest1803
[13:29] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:30] * ngoswami (~ngoswami@121.244.87.116) Quit (Read error: Connection reset by peer)
[13:30] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[13:32] * penguinRaider (~KiKo@69.163.33.182) has joined #ceph
[13:34] * Guest1803 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:35] * swami1 (~swami@49.38.0.153) has joined #ceph
[13:38] * itamarl is now known as Guest1804
[13:38] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:40] * tserong (~tserong@203-214-92-220.dyn.iinet.net.au) has joined #ceph
[13:41] * Guest1804 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:41] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[13:43] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[13:44] * itamarl is now known as Guest1805
[13:44] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:44] * wjw-freebsd (~wjw@smtp.digiware.nl) Quit (Ping timeout: 480 seconds)
[13:44] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[13:46] * Guest1805 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:47] * Meths (~meths@95.151.244.156) has joined #ceph
[13:48] * itamarl is now known as Guest1806
[13:48] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:51] * itamarl is now known as Guest1808
[13:51] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[13:51] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:52] * Guest1806 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:52] * Meths (~meths@95.151.244.156) has joined #ceph
[13:56] * Guest1808 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[13:56] * itamarl is now known as Guest1809
[13:56] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[13:58] * flisky (~Thunderbi@210.12.157.94) Quit (Ping timeout: 480 seconds)
[13:58] * shyu (~Frank@218.241.172.114) Quit (Ping timeout: 480 seconds)
[13:58] * Linkmark (~Linkmark@252.146-78-194.adsl-static.isp.belgacom.be) has joined #ceph
[13:59] * Guest1809 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:00] * itamarl is now known as Guest1810
[14:00] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:00] * kuku (~kuku@112.203.51.69) Quit (Read error: Connection reset by peer)
[14:01] * kuku (~kuku@112.203.51.69) has joined #ceph
[14:03] * lkoranda (~lkoranda@nat-pool-brq-t.redhat.com) has joined #ceph
[14:03] * kuku (~kuku@112.203.51.69) Quit (Read error: Connection reset by peer)
[14:03] * kuku (~kuku@112.203.51.69) has joined #ceph
[14:04] <Linkmark> Hi guy's, we have a ceph version 0.72 running with 24 1TB osd's running. 4 nodes with each 6 OSD's, one of the nodes rebooted but ceph didn't come up after the reboot. Now we have 10 OSD's down because they got full (100% full after re-balance because of node reboot)) and just crashed because of that. Restarting ceph on the rebooted node rebalances our data but with all the osd's down these are also getting full. What is the b
[14:04] <Linkmark> est way to get trough this? Any guides or documents?
[14:04] * Guest1810 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:04] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[14:05] * Meths (~meths@95.151.244.156) has joined #ceph
[14:06] * itamarl is now known as Guest1812
[14:06] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:06] <Be-El> Linkmark: disable backfill and bring back all osds
[14:06] <Be-El> (and update to a non-ancient version afterwards... )
[14:08] * Guest1812 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:09] * georgem (~Adium@206.108.127.16) has joined #ceph
[14:09] <Linkmark> Be-El, ok thanks, is it that 'simpel' ?
[14:09] <Be-El> Linkmark: it's the first attempt. if it is not going to work, you may need more complex solutions
[14:09] <Linkmark> Good point about the ancient version .. company policy you know .. if it works, why break it :/
[14:11] <Linkmark> Be-El, ok looking into it. Thanks!
[14:11] * itamarl is now known as Guest1813
[14:11] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:14] * Guest1813 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:15] * itamarl is now known as Guest1814
[14:15] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:19] * itamarl is now known as Guest1816
[14:19] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:19] * Guest1814 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:20] * liamchou (~liamchou@183.45.18.103) Quit (Quit: Leaving)
[14:23] * Guest1816 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:26] * kuku (~kuku@112.203.51.69) Quit (Remote host closed the connection)
[14:26] * karnan (~karnan@121.244.87.117) Quit (Remote host closed the connection)
[14:27] * rogst (~Lunk2@06SAAET0F.tor-irc.dnsbl.oftc.net) has joined #ceph
[14:27] * itamarl is now known as Guest1818
[14:27] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:30] * Guest1818 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:30] * kuku (~kuku@112.203.51.69) has joined #ceph
[14:31] * gauravbafna (~gauravbaf@49.38.1.125) Quit (Remote host closed the connection)
[14:34] * itamarl is now known as Guest1819
[14:34] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:37] * Guest1819 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:37] * itamarl is now known as Guest1820
[14:37] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:40] * bjornar_ (~bjornar@2.150.52.131.tmi.telenormobil.no) has joined #ceph
[14:42] * Guest1820 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:42] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[14:46] * kuku (~kuku@112.203.51.69) Quit (Remote host closed the connection)
[14:47] * kuku (~kuku@112.203.51.69) has joined #ceph
[14:47] * itamarl is now known as Guest1821
[14:47] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:49] * kuku (~kuku@112.203.51.69) Quit (Remote host closed the connection)
[14:49] * kuku (~kuku@112.203.51.69) has joined #ceph
[14:50] * bvi (~Bastiaan@185.56.32.1) has joined #ceph
[14:50] * kuku (~kuku@112.203.51.69) Quit (Read error: Connection reset by peer)
[14:51] * kuku (~kuku@112.203.51.69) has joined #ceph
[14:52] * Guest1821 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:54] * itamarl is now known as Guest1822
[14:54] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[14:55] * bjornar_ (~bjornar@2.150.52.131.tmi.telenormobil.no) Quit (Ping timeout: 480 seconds)
[14:55] * Guest1822 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[14:56] * sebastian-w_ (~quassel@212.218.8.138) has joined #ceph
[14:56] * rogst (~Lunk2@06SAAET0F.tor-irc.dnsbl.oftc.net) Quit ()
[14:57] * kawa2014 (~kawa@89.184.114.246) Quit (Ping timeout: 480 seconds)
[14:58] * sebastian-w (~quassel@212.218.8.139) Quit (Ping timeout: 480 seconds)
[14:59] * georgem (~Adium@206.108.127.16) has joined #ceph
[15:00] * dyasny (~dyasny@cable-192.222.152.136.electronicbox.net) has joined #ceph
[15:00] * DanFoster (~Daniel@office.34sp.com) Quit (Quit: Leaving)
[15:01] * DanFoster (~Daniel@2a00:1ee0:3:1337:8c56:821a:667:cc0c) has joined #ceph
[15:03] * itamarl is now known as Guest1823
[15:03] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:04] * sebastian-w (~quassel@212.218.8.138) has joined #ceph
[15:06] * Guest1823 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:07] * kuku (~kuku@112.203.51.69) Quit (Remote host closed the connection)
[15:07] * sebastian-w_ (~quassel@212.218.8.138) Quit (Ping timeout: 480 seconds)
[15:08] * kawa2014 (~kawa@212.110.41.244) has joined #ceph
[15:10] * koszik (~koszik@88.151.96.5) has joined #ceph
[15:10] * rdias (~rdias@2001:8a0:749a:d01:f525:c32e:c1a4:34a0) Quit (Ping timeout: 480 seconds)
[15:11] * itamarl is now known as Guest1824
[15:11] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:11] * sebastian-w_ (~quassel@212.218.8.139) has joined #ceph
[15:11] * nagyz (~ubuntu@pat.zurich.ibm.com) has joined #ceph
[15:11] * nagyz (~ubuntu@pat.zurich.ibm.com) Quit (Read error: Connection reset by peer)
[15:13] * sebastian-w (~quassel@212.218.8.138) Quit (Read error: Connection reset by peer)
[15:13] * kuku (~kuku@112.203.51.69) has joined #ceph
[15:14] * allaok (~allaok@machine107.orange-labs.com) has left #ceph
[15:14] * vata1 (~vata@207.96.182.162) has joined #ceph
[15:15] * Guest1824 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:15] * kuku (~kuku@112.203.51.69) Quit (Read error: Connection reset by peer)
[15:15] * itamarl is now known as Guest1825
[15:15] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:15] * kuku (~kuku@112.203.51.69) has joined #ceph
[15:16] * nagyz (~ubuntu@pat.zurich.ibm.com) has joined #ceph
[15:16] * nagyz (~ubuntu@pat.zurich.ibm.com) Quit (Read error: Connection reset by peer)
[15:19] * Guest1825 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:19] * tserong (~tserong@203-214-92-220.dyn.iinet.net.au) Quit (Quit: Leaving)
[15:20] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) has joined #ceph
[15:23] * itamarl is now known as Guest1827
[15:23] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:26] * Guest1827 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:27] * itamarl is now known as Guest1829
[15:27] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:31] * Guest1829 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:31] * itamarl is now known as Guest1831
[15:31] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:33] * Zeis (~Grimhound@d4.00.39a9.ip4.static.sl-reverse.com) has joined #ceph
[15:35] <Be-El> anyone around who is using intel p3700 nvm ssds as pool storage (in addition to journals)?
[15:35] * gauravbafna (~gauravbaf@122.172.196.208) has joined #ceph
[15:35] * Guest1831 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:36] * gauravbafna (~gauravbaf@122.172.196.208) Quit (Remote host closed the connection)
[15:36] * gauravbafna (~gauravbaf@122.172.196.208) has joined #ceph
[15:37] * tserong (~tserong@203-214-92-220.dyn.iinet.net.au) has joined #ceph
[15:38] * gauravbafna (~gauravbaf@122.172.196.208) Quit (Remote host closed the connection)
[15:38] * itamarl is now known as Guest1832
[15:38] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:39] * Guest1832 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:40] * gauravbafna (~gauravbaf@122.172.196.208) has joined #ceph
[15:41] * vata1 (~vata@207.96.182.162) Quit (Quit: Leaving.)
[15:42] * vata1 (~vata@207.96.182.162) has joined #ceph
[15:44] * itamarl is now known as Guest1833
[15:44] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:46] * Guest1833 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:47] <The_Ball> What's the recommended max for OSD space usage? 75-80%?
[15:48] * itamarl is now known as Guest1836
[15:48] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:48] <The_Ball> I guess with a larger number of OSDs you could go higher as a few OSDs going out wouldn't add much to the remaining OSDs
[15:51] * kawa2014 (~kawa@212.110.41.244) Quit (Ping timeout: 480 seconds)
[15:52] * itamarl is now known as Guest1838
[15:52] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[15:52] * Guest1836 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:52] * Kurt (~Adium@2001:628:1:5:54c9:c124:c356:acda) has joined #ceph
[15:55] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[15:56] * Guest1838 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[15:56] * mewald (~Adium@185.80.187.212) has joined #ceph
[15:57] * itamarl is now known as Guest1839
[15:57] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:00] * Guest1839 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:00] * kawa2014 (~kawa@89.184.114.246) has joined #ceph
[16:01] * saintpablos (~saintpabl@2.106.149.41) has joined #ceph
[16:01] <Be-El> The_Ball: you need to consider two things: uneven data distribution (up to 15% difference), and your failure model
[16:01] * analbeard (~shw@host109-150-56-204.range109-150.btcentralplus.com) has joined #ceph
[16:01] <Be-El> The_Ball: if your failure model allows to complete hosts to fail, you need to have enough space on the other hosts to hold all the data from the two failed hosts
[16:01] <Be-El> s/to/two/
[16:02] * vikhyat (~vumrao@49.248.193.82) Quit (Quit: Leaving)
[16:02] * Zeis (~Grimhound@7V7AAG1PP.tor-irc.dnsbl.oftc.net) Quit ()
[16:03] * itamarl is now known as Guest1841
[16:03] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:06] <The_Ball> sure that makes sense
[16:07] * Guest1841 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:07] * itamarl is now known as Guest1843
[16:07] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:07] * saintpablo (~saintpabl@gw01.mhitp.dk) Quit (Ping timeout: 480 seconds)
[16:10] * saintpablos (~saintpabl@2.106.149.41) Quit (Ping timeout: 480 seconds)
[16:10] * yanzheng (~zhyan@125.70.20.240) Quit (Quit: This computer has gone to sleep)
[16:11] * Guest1843 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:13] * itamarl is now known as Guest1845
[16:13] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:15] * Guest1845 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:16] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[16:19] * itamarl is now known as Guest1847
[16:19] * kuku (~kuku@112.203.51.69) Quit (Read error: Connection reset by peer)
[16:19] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:20] * kuku (~kuku@112.203.51.69) has joined #ceph
[16:21] * Guest1847 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:23] * itamarl is now known as Guest1848
[16:23] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:24] * kuku (~kuku@112.203.51.69) Quit (Remote host closed the connection)
[16:27] * Guest1848 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:28] * analbeard (~shw@host109-150-56-204.range109-150.btcentralplus.com) Quit (Quit: Leaving.)
[16:29] * swami1 (~swami@49.38.0.153) Quit (Quit: Leaving.)
[16:32] * flisky (~Thunderbi@106.37.236.191) has joined #ceph
[16:32] * itamarl is now known as Guest1850
[16:32] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:34] * Guest1850 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:40] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[16:42] * mewald (~Adium@185.80.187.212) Quit (Quit: Leaving.)
[16:43] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[16:46] * rraja (~rraja@121.244.87.117) Quit (Quit: Leaving)
[16:46] * itamarl is now known as Guest1851
[16:46] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[16:47] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[16:49] * flisky (~Thunderbi@106.37.236.191) Quit (Ping timeout: 480 seconds)
[16:50] * gwinger (~gwinger@2a02:8108:42c0:a68:9142:ec81:e442:f510) Quit (Quit: Verlassend)
[16:51] * Guest1851 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[16:55] * Hemanth (~hkumar_@121.244.87.117) Quit (Ping timeout: 480 seconds)
[17:02] * itamarl is now known as Guest1853
[17:02] * itamarl (~itamarl@194.90.7.244) has joined #ceph
[17:07] * Guest1853 (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[17:08] * mewald (~Adium@62.214.139.242) has joined #ceph
[17:11] * toastyde1th (~toast@pool-71-255-253-39.washdc.fios.verizon.net) has joined #ceph
[17:16] * newbie (~kvirc@host217-114-156-249.pppoe.mark-itt.net) has joined #ceph
[17:17] * toastydeath (~toast@pool-71-255-253-39.washdc.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[17:18] * mewald1 (~Adium@82.113.99.252) has joined #ceph
[17:22] * Linkmark (~Linkmark@252.146-78-194.adsl-static.isp.belgacom.be) Quit (Quit: Leaving)
[17:23] * mewald (~Adium@62.214.139.242) Quit (Ping timeout: 480 seconds)
[17:27] * itamarl (~itamarl@194.90.7.244) Quit (Ping timeout: 480 seconds)
[17:28] * evelu (~erwan@46.231.131.178) Quit (Ping timeout: 480 seconds)
[17:29] * mewald (~Adium@89.15.237.104) has joined #ceph
[17:32] * mewald1 (~Adium@82.113.99.252) Quit (Ping timeout: 480 seconds)
[17:42] * bvi (~Bastiaan@185.56.32.1) Quit (Quit: Leaving)
[17:42] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[17:42] * TMM (~hp@185.5.121.201) Quit (Quit: Ex-Chat)
[17:42] * Meths (~meths@95.151.244.156) has joined #ceph
[17:44] * swami1 (~swami@27.7.164.158) has joined #ceph
[17:44] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[17:47] * Meths (~meths@95.151.244.156) has joined #ceph
[18:00] * Racpatel (~Racpatel@c-73-170-66-165.hsd1.ca.comcast.net) has joined #ceph
[18:01] * mykola (~Mikolaj@91.245.79.99) has joined #ceph
[18:03] * gauravbafna (~gauravbaf@122.172.196.208) Quit (Remote host closed the connection)
[18:03] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[18:03] * branto (~branto@ip-78-102-208-181.net.upcbroadband.cz) Quit (Quit: Leaving.)
[18:04] * Meths (~meths@95.151.244.156) has joined #ceph
[18:06] * kawa2014 (~kawa@89.184.114.246) Quit (Quit: Leaving)
[18:07] * ade (~abradshaw@dslb-088-075-188-119.088.075.pools.vodafone-ip.de) Quit (Quit: Too sexy for his shirt)
[18:11] * mewald (~Adium@89.15.237.104) Quit (Quit: Leaving.)
[18:11] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[18:11] * Meths (~meths@95.151.244.156) has joined #ceph
[18:12] * dgurtner (~dgurtner@84-73-130-19.dclient.hispeed.ch) Quit (Ping timeout: 480 seconds)
[18:15] * swami1 (~swami@27.7.164.158) Quit (Quit: Leaving.)
[18:15] * gauravbafna (~gauravbaf@122.172.196.208) has joined #ceph
[18:15] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: This computer has gone to sleep)
[18:20] * haplo37 (~haplo37@199.91.185.156) has joined #ceph
[18:26] * mewald (~Adium@62.214.139.242) has joined #ceph
[18:36] * ceph-devel (uid171189@id-171189.highgate.irccloud.com) Quit (Quit: Connection closed for inactivity)
[18:44] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[18:44] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:9901:29b4:b6c5:3fa9) Quit (Ping timeout: 480 seconds)
[18:47] * mewald (~Adium@62.214.139.242) Quit (Quit: Leaving.)
[18:54] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) has left #ceph
[19:03] * gauravba_ (~gauravbaf@223.227.137.82) has joined #ceph
[19:05] * gauravbafna (~gauravbaf@122.172.196.208) Quit (Ping timeout: 480 seconds)
[19:11] * Hemanth (~hkumar_@103.228.221.137) has joined #ceph
[19:19] * Skaag (~lunix@cpe-172-91-77-84.socal.res.rr.com) Quit (Quit: Leaving.)
[19:23] * wjw-freebsd (~wjw@smtp.digiware.nl) has joined #ceph
[19:23] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[19:24] * Meths (~meths@95.151.244.156) has joined #ceph
[19:30] * kiasyn (~notmyname@nl3x.mullvad.net) has joined #ceph
[19:30] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[19:31] * Meths (~meths@95.151.244.156) has joined #ceph
[19:31] * debian112 (~bcolbert@c-73-184-103-26.hsd1.ga.comcast.net) Quit (Ping timeout: 480 seconds)
[19:32] * bjornar_ (~bjornar@ti0099a430-0410.bb.online.no) has joined #ceph
[19:33] * Racpatel (~Racpatel@c-73-170-66-165.hsd1.ca.comcast.net) Quit (Quit: Leaving)
[19:35] <koszik> _calc_past_interval_range: already have past intervals back to 27149 - this is on an osd that should have the replica of the pg, but the directory holding the pg on that osd is completely empty. is this normal or a bug?
[19:35] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[19:36] * Meths (~meths@95.151.244.156) has joined #ceph
[19:38] <TheSov> anyone around know why an empty RBD pool has objects in it?
[19:38] * BrianA (~BrianA@c-24-130-77-245.hsd1.ca.comcast.net) has joined #ceph
[19:39] <koszik> rbd 2 0 0 5433G 1
[19:39] <koszik> like this?
[19:39] <koszik> rbd ls --pool rbd is also returning nothing
[19:40] * mewald (~Adium@89.15.237.104) has joined #ceph
[19:40] * swami1 (~swami@27.7.164.158) has joined #ceph
[19:41] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) has joined #ceph
[19:44] * debian112 (~bcolbert@c-73-184-103-26.hsd1.ga.comcast.net) has joined #ceph
[19:47] <TheSov> yes
[19:47] <TheSov> my first thought is, orphaned objects
[19:47] <TheSov> but i though ceph takes care of that
[19:48] <TheSov> root@ceph-1:/home/user# ceph df
[19:48] <TheSov> GLOBAL:
[19:48] <TheSov> SIZE AVAIL RAW USED %RAW USED
[19:48] <TheSov> 8379G 7876G 502G 6.00
[19:48] <TheSov> POOLS:
[19:48] <TheSov> NAME ID USED %USED MAX AVAIL OBJECTS
[19:48] <TheSov> rbd 0 3552M 0.12 2598G 889
[19:48] <TheSov> pmx-storage 1 29423M 1.03 2598G 8690
[19:48] <TheSov> pmx-dual 2 202G 4.84 3897G 53205
[19:48] <TheSov> root@ceph-1:/home/user#
[19:48] <TheSov> rbd in this case has no images 889 objects
[19:49] * mykola (~Mikolaj@91.245.79.99) Quit (Read error: Connection reset by peer)
[19:49] <TheSov> any thoughts?
[19:50] * mykola (~Mikolaj@91.245.79.99) has joined #ceph
[19:50] <TheSov> koszik, ?
[19:50] <koszik> i don't think you want to hear my thoughts after spending several days just to be able to access the data stored on my ceph cluster...
[19:50] <TheSov> lol
[19:50] <TheSov> that bad?
[19:50] <koszik> yes
[19:50] <TheSov> if you dont mind, what happened?
[19:50] <koszik> and still not resolved
[19:50] <koszik> not sure
[19:51] <TheSov> hammer?
[19:51] <koszik> one of the nodes rebooted, then all hell broke loose
[19:51] <TheSov> what release are you on?
[19:51] <koszik> was firefly at the beginning, but upgraded through hammer to jewel in the process, hoping for the problem to go away
[19:51] <koszik> (it didn't)
[19:51] <TheSov> oh i started my home cluster at jewel, and my work cluster is hammer
[19:51] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[19:52] <TheSov> how many monitors you got?
[19:52] * Meths (~meths@95.151.244.156) has joined #ceph
[19:52] <koszik> 4 at the moment
[19:52] <TheSov> ...
[19:52] <TheSov> is one down?
[19:52] <koszik> no
[19:52] <TheSov> u cant have an even number of monitors
[19:53] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[19:53] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[19:53] <TheSov> it goes 1, 3, 5, 7, 9 etc
[19:53] <TheSov> with the optimal number being 3
[19:54] <koszik> i never lost more than 1 monitor so i don't think it matters that much in this cse
[19:54] <koszik> they always were able to decide if they have quorum or not
[19:54] <koszik> but sure, that's something that needs to be fixed, after i'm able to access all my rbds
[19:54] <TheSov> im pretty sure your problem may be related to that
[19:55] * nils_ (~nils_@doomstreet.collins.kg) Quit (Quit: This computer has gone to sleep)
[19:55] <koszik> well.. i had a bunch of unfound objects (after that node reboot), which cannot be caused by the wrong number of monitors
[19:56] <koszik> then the osds having the unfound objects just failed to handle any messages, not even a tell osd.$x version
[19:57] <koszik> it would just hang there
[19:57] * Meths (~meths@95.151.244.156) has joined #ceph
[19:57] <TheSov> actually monitors are the ones that keep track of what object belongs to what
[19:58] <TheSov> otherwise its just a bunch of files on a hard drive
[19:58] <TheSov> the "data map" resides on monitors and is given to clients
[19:59] <TheSov> then the clients can request their own data, but any writes of new data has to hit the monitors again
[19:59] * kiasyn (~notmyname@nl3x.mullvad.net) Quit ()
[20:00] <TheSov> funny enough i was actually pretty sure my orphaned objects exist because i was fucking with monitors
[20:00] <koszik> i understand monitors are pretty importand and i also think i understand why an even number of them is bad (if you partition your network they won't know if they're in majority)
[20:00] <koszik> but i don't think that should cause any of the issues i had/have
[20:00] * derjohn_mobi (~aj@fw.gkh-setu.de) Quit (Ping timeout: 480 seconds)
[20:01] <TheSov> you are correct as long as you maintain connectivity it should work fine.
[20:01] <koszik> for example, there's a pg that is active+recovery_wait+undersized+degraded+remapped ; it's on osds 10,5, 10 being the primary, it has all the data, and on 5 the pg directory is empty
[20:02] <TheSov> but i have seen large scale deployments, and no joke having more than 3 monitors; shit starts to get quirky
[20:02] <TheSov> remember all writes have to hit all monitors all the time
[20:02] <koszik> i have restarted it a hundred times but it just won't copy the data
[20:02] <TheSov> wow thats odd
[20:02] <TheSov> do you have a backup of your data?
[20:02] <koszik> they even peer and everything
[20:03] <TheSov> and whats your pg count vs # of osds?
[20:03] <koszik> the important data has backups yes
[20:03] * nhm (~nhm@c-50-171-139-246.hsd1.mn.comcast.net) Quit (Ping timeout: 480 seconds)
[20:03] <koszik> 51 osds and ~4300 pgs
[20:04] <TheSov> well
[20:04] <TheSov> your at under 100 pg's per osd
[20:04] <TheSov> but thats not that bad
[20:05] <TheSov> also try to keep pg's a multiple of 2
[20:05] <TheSov> closest is 4096
[20:06] <TheSov> perhaps you should increase your pg count to 8192 once you fix your issues
[20:07] <TheSov> man dude it seems like you have a lot of data
[20:08] <koszik> a few tbs, yes
[20:08] <koszik> ceph looked like such a good thing, it has great flexibility
[20:08] <koszik> but that comes at a cost of unbearable complexity, especially if something goes wrong
[20:09] <koszik> i
[20:13] <IcePic> trust me, big vendor storage systems are the same, when you do something "unexpected" like fill 85% of them or so, they also act up in hard-to-diagnose ways.
[20:13] * vata (~vata@cable-173.246.3-246.ebox.ca) Quit (Ping timeout: 480 seconds)
[20:14] <koszik> if only IT would work without having to keep any data...
[20:16] <IcePic> not that, but it would at least be something if you could pay your way out of oddball scenarios it would be something, then you could say "well, we went cheap and now it sucks to be us", but in my experience, the big $$$ disk systems that claim to scale arent much simpler for me, and doesn't lose less data, they just do it in other ways.
[20:16] <IcePic> for me that has to tell users they lost data, they dont care I had some contract that says "if stuff happens, blame us for a while" or so, lost work is lost work anyhow.
[20:17] <koszik> looks like osds 4 and 43 contain some stray content from this pg. is it the cause for failing this pg to have a normal state, or not? i'll probably know after some more hours of debugging... but first let's eat some.
[20:17] <koszik> only simplicity works, while i had a single storage server with drbd i had almost zero issues, but that didn't scale too well
[20:22] * mgolub (~Mikolaj@193.93.217.49) has joined #ceph
[20:22] * mewald (~Adium@89.15.237.104) Quit (Read error: Connection reset by peer)
[20:22] * mewald (~Adium@82.113.99.10) has joined #ceph
[20:23] <koszik> of course, i already distort my memories, because thinking a little harder i remember quite some issues, but nothing of this scale (which is kinda expected)
[20:24] * xarses_ (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[20:25] <TheSov> what do you find so complex about ceph?
[20:25] * vata (~vata@cable-173.246.3-246.ebox.ca) has joined #ceph
[20:25] <TheSov> i look it and i see a bunch of simple systems connected together
[20:25] <TheSov> the most complex peice is the monitor
[20:26] <koszik> i think the osd is the piece that gives me a hard time right now, i have no idea why doesn't it do what it's supposed to
[20:26] <TheSov> im just trying to figure out if i can manually delete these objects without affecting anything
[20:26] * mykola (~Mikolaj@91.245.79.99) Quit (Ping timeout: 480 seconds)
[20:26] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[20:26] <koszik> everything is up and connected, yet it stays at active+undersized+degraded+remapped+wait_backfill
[20:26] <koszik> also a lot of active+recovery_wait+degraded
[20:27] <koszik> it was worse, but then i kept restarting osds and was getting better, but that method is of no use anymore
[20:31] <georgem> koszik: did you change the weight of your osds?
[20:32] <TheSov> koszik show us your "ceph osd tree"
[20:32] <TheSov> and are your disks different sizes?
[20:35] * jargonmonk (~jmnk@203.187.233.75) has joined #ceph
[20:37] * Meths (~meths@95.151.244.156) has joined #ceph
[20:40] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[20:40] * Meths (~meths@95.151.244.156) Quit (Read error: Connection reset by peer)
[20:42] * Meths (~meths@95.151.244.156) has joined #ceph
[20:45] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[20:46] * derjohn_mobi (~aj@p578b6aa1.dip0.t-ipconnect.de) has joined #ceph
[20:48] <jargonmonk> Hello!
[20:50] * jargonmonk is now known as jmnk
[20:52] * Discovery (~Discovery@178.239.49.68) has joined #ceph
[20:54] * mewald (~Adium@82.113.99.10) Quit (Quit: Leaving.)
[20:54] * neurodrone (~neurodron@162.243.191.67) has joined #ceph
[20:56] * Hemanth (~hkumar_@103.228.221.137) Quit (Quit: Leaving)
[20:59] * praveen (~praveen@121.244.155.12) has joined #ceph
[21:00] * mewald (~Adium@82.113.99.10) has joined #ceph
[21:04] <praveen> Hi anybody there
[21:05] * praveen (~praveen@121.244.155.12) Quit (Remote host closed the connection)
[21:06] * bvi (~Bastiaan@102-117-145-85.ftth.glasoperator.nl) has joined #ceph
[21:18] * niknakpaddywak (~xander.ni@outbound.lax.demandmedia.com) Quit (Ping timeout: 480 seconds)
[21:20] * moon (~moon@tunnel.greenhost.nl) has joined #ceph
[21:24] * rotbeard (~redbeard@185.32.80.238) Quit (Quit: Leaving)
[21:31] * Jeffrey4l_ (~Jeffrey@119.251.240.50) has joined #ceph
[21:34] * toMeloos (~toMeloos@53568B3D.cm-6-7c.dynamic.ziggo.nl) has joined #ceph
[21:35] * swami1 (~swami@27.7.164.158) Quit (Quit: Leaving.)
[21:38] * mewald (~Adium@82.113.99.10) Quit (Quit: Leaving.)
[21:38] * Jeffrey4l (~Jeffrey@121.16.246.42) Quit (Ping timeout: 480 seconds)
[21:38] * praveen (~praveen@122.172.223.47) has joined #ceph
[21:42] * rendar (~I@host47-183-dynamic.7-87-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[21:42] * gauravba_ (~gauravbaf@223.227.137.82) Quit (Remote host closed the connection)
[21:43] * mgolub (~Mikolaj@193.93.217.49) Quit (Quit: away)
[21:43] * gauravbafna (~gauravbaf@223.227.137.82) has joined #ceph
[21:51] * derjohn_mobi (~aj@p578b6aa1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[21:54] * shaunm (~shaunm@74.83.215.100) has joined #ceph
[21:55] * gauravbafna (~gauravbaf@223.227.137.82) Quit (Remote host closed the connection)
[21:55] * gauravbafna (~gauravbaf@223.227.137.82) has joined #ceph
[22:08] * rendar (~I@host47-183-dynamic.7-87-r.retail.telecomitalia.it) has joined #ceph
[22:14] * newbie (~kvirc@host217-114-156-249.pppoe.mark-itt.net) Quit (Ping timeout: 480 seconds)
[22:19] * Discovery (~Discovery@178.239.49.68) Quit (Read error: Connection reset by peer)
[22:20] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[22:26] * jargonmonk (~jmnk@123.201.112.108) has joined #ceph
[22:28] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:586e:fbca:9eb1:4ebb) has joined #ceph
[22:29] * moon (~moon@tunnel.greenhost.nl) Quit (Ping timeout: 480 seconds)
[22:31] * madkiss (~madkiss@2001:6f8:12c3:f00f:cd6b:c321:3774:edbf) Quit (Ping timeout: 480 seconds)
[22:31] * gauravbafna (~gauravbaf@223.227.137.82) Quit (Remote host closed the connection)
[22:33] * jmnk (~jmnk@203.187.233.75) Quit (Ping timeout: 480 seconds)
[22:36] * madkiss1 (~madkiss@2001:6f8:12c3:f00f:586e:fbca:9eb1:4ebb) Quit (Read error: Connection reset by peer)
[22:36] <koszik> TheSov, yes, they're of different sizes, not weighted correctly yet
[22:37] * madkiss (~madkiss@2001:6f8:12c3:f00f:586e:fbca:9eb1:4ebb) has joined #ceph
[23:00] * theTrav (~theTrav@CPE-124-188-218-238.sfcz1.cht.bigpond.net.au) has joined #ceph
[23:04] * pfactum is now known as post-factum
[23:07] * TomasCZ (~TomasCZ@yes.tenlab.net) has joined #ceph
[23:09] * neurodrone (~neurodron@162.243.191.67) Quit (Quit: neurodrone)
[23:20] * georgem (~Adium@24.114.57.36) has joined #ceph
[23:21] * georgem (~Adium@24.114.57.36) Quit ()
[23:21] * georgem (~Adium@206.108.127.16) has joined #ceph
[23:32] * haplo37 (~haplo37@199.91.185.156) Quit (Remote host closed the connection)
[23:34] * realitysandwich (~perry@b2b-78-94-59-114.unitymedia.biz) has joined #ceph
[23:35] * bvi (~Bastiaan@102-117-145-85.ftth.glasoperator.nl) Quit (Ping timeout: 480 seconds)
[23:35] <realitysandwich> any idea why when I am writing data to my ceph pool, one osd per server (3 machines total with 7 osds each) will have about 50% total usage according to iostat while the rest are around 2%?
[23:38] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[23:39] * DV_ (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[23:39] * georgem (~Adium@206.108.127.16) Quit (Quit: Leaving.)
[23:42] * bjornar_ (~bjornar@ti0099a430-0410.bb.online.no) Quit (Ping timeout: 480 seconds)
[23:49] * albertom (~albertom@134.134.139.83) has joined #ceph
[23:56] * hellertime (~Adium@a72-246-0-10.deploy.akamaitechnologies.com) has joined #ceph
[23:56] <hellertime> Is the cephx key algorithm documented anywhere?

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.