#ceph IRC Log

Index

IRC Log for 2016-06-15

Timestamps are in GMT/BST.

[0:01] <jamespd_> gwinger: possibly. what's up?
[0:01] <SamYaple> its normally better to ask the question rather than ask to ask the question
[0:02] * _sileht is now known as sileht
[0:03] <gwinger> have serious problems on creating a secondary cloudstack nfs
[0:03] <gwinger> I want to use cep for that
[0:04] <gwinger> Tried first the thing with crush mapping special SSD's to the first pool for primary
[0:04] <gwinger> the second pool for secondary cloudstack storage
[0:05] <gwinger> cause I have 3 Poweredges with each has 2 SSD'S 960 TB which I want to have cep using as a pool or as a separate cluster
[0:05] <gwinger> then I have 4 SATA each machine with each has 4 TB
[0:06] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[0:06] <gwinger> Problem is, that I could successfully create a primary cloudstack Object Storage but not the secondary
[0:06] * squizzi (~squizzi@107.13.31.195) Quit (Quit: bye)
[0:06] <gwinger> So I think it is not the right way
[0:06] <gwinger> perhaps
[0:07] <gwinger> So trying to build 2 cep clusters on the hardware
[0:07] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[0:07] <gwinger> host 0 - host2 to have some replication
[0:07] <gwinger> on the 3 machines
[0:08] <gwinger> when I use the 3 * 4 * 4TB for a RDB with nfs
[0:08] <gwinger> But do I need 2 cep monitors then on each Powered to run 2 different ceps clusters there?
[0:09] <gwinger> Anything clear?
[0:09] <gwinger> Sorry, my english
[0:13] * DV (~veillard@2001:41d0:a:f29f::1) Quit (Remote host closed the connection)
[0:14] * ira (~ira@c-24-34-255-34.hsd1.ma.comcast.net) Quit (Quit: Leaving)
[0:14] <gwinger> I find no docs anywhere explaining how to configure out to have 2 separate ceps clusters on the same hardware.
[0:15] * DV (~veillard@2001:41d0:a:f29f::1) has joined #ceph
[0:15] <gwinger> Don't anybody know?
[0:18] <Kvisle> it looks to me like the ceph-mon in the 10.2.1-0.el7.x86_64 packages is leaking memory - just upgraded, and it's memory footprint is increasing linearly
[0:19] * mattbenjamin (~mbenjamin@12.118.3.106) Quit (Ping timeout: 480 seconds)
[0:19] <Kvisle> http://imgur.com/JRKbR3I
[0:19] <SamYaple> gwinger: you can do it, but its not recommended to do so. name them different cluster names
[0:20] <SamYaple> gwinger: you also have ot update all the ports so there are no conflicts
[0:20] <gwinger> ok, not good news
[0:21] * gregmark (~Adium@68.87.42.115) Quit (Quit: Leaving.)
[0:21] <gwinger> I asked myself wheater it is possible to do a separation with pooling inside a cluster
[0:21] <gwinger> All the physical storage devices together in one cluster
[0:22] <gwinger> then create 2 pools and adapt the crush map so that the primary pool only stores on the SSD's
[0:22] <gwinger> and the secondary only stores to the SATA disks
[0:23] <gwinger> I got this working and the ceps tree was quite showing everything is correct
[0:24] <gwinger> Then I added the ssl pool (SSD's) as primary storage in Cloudstack
[0:24] <gwinger> But Cloudstack showed the complete cluster size
[0:25] * poller (~LRWerewol@7V7AAF7Y1.tor-irc.dnsbl.oftc.net) has joined #ceph
[0:25] <gwinger> which is corrects on one hand, but the primary ssl pool should only use 5.8 TB not 48 TB (both pools added size)
[0:26] <gwinger> This was fly first problem...
[0:26] * darkfader (~floh@88.79.251.60) Quit (Ping timeout: 480 seconds)
[0:26] <gwinger> Think, that the wrong information disturbs the quota
[0:26] * jclm (~jclm@rrcs-69-193-93-163.nys.biz.rr.com) has joined #ceph
[0:27] <gwinger> Second thing what I don't really understand is, that I wasn't able to map and mount a RBD on the second pool (secondary)
[0:28] <gwinger> I configured an nfs on the RBD
[0:28] <gwinger> It was correctly showing up in /dev/rbd0/secondary/nfs
[0:28] <gwinger> after mapping
[0:29] <gwinger> I mapped it on each of the three Poweredges
[0:29] <gwinger> (UBUNTU 16
[0:29] <gwinger> And it was exported on /mnt/secondary
[0:30] * andreww (~xarses@ip-64-134-222-177.public.wayport.net) Quit (Remote host closed the connection)
[0:30] * xarses (~xarses@ip-64-134-222-177.public.wayport.net) has joined #ceph
[0:30] <gwinger> So I was able to do a touch my_file and the file was saved in UBUNTU
[0:30] <EthanL> Can anyone help me get my ceph cluster set up? I am trying to get it setup following these instructions: http://docs.ceph.com/docs/jewel/install/manual-deployment/ but it fails when I try "ceph osd lspools"
[0:30] <gwinger> But it doesn't show up on the two other hosts
[0:31] * darkfader (~floh@88.79.251.60) has joined #ceph
[0:31] <EthanL> I am trying to get a ceph cluster up so I can mount a drive as a bootable drive for one of the systems
[0:31] <gwinger> I can't understand why this happens
[0:31] <SamYaple> gwinger: as far as i know there is know way to filter the cluster size when you split it up like that.
[0:32] <gwinger> no way?
[0:32] * vata (~vata@207.96.182.162) Quit (Quit: Leaving.)
[0:32] <SamYaple> gwinger: are you saying that you mapped a single rbd to multiple hosts?
[0:32] <gwinger> maybe yes
[0:32] <SamYaple> you cant do that
[0:32] <gwinger> ok
[0:32] <SamYaple> well let me rephrase that
[0:32] <SamYaple> you _can_ do that but you probably dont want to
[0:33] <SamYaple> you cant have a filesystem mounted on several OS at the same time
[0:33] <SamYaple> you have to setup the rbd on one host and share it out with nfs on other hosts
[0:33] <blizzow> SamYaple: What do you mean?? I have a single RBD pool mapped as a storage pool for all my libvirt hypervisors.
[0:33] <SamYaple> blizzow: rbd pool, not indivdual rbd
[0:33] <gwinger> hmm
[0:33] * mivaho (~quassel@2001:983:eeb4:1:c0de:69ff:fe2f:5599) Quit (Quit: No Ping reply in 180 seconds.)
[0:34] * mivaho (~quassel@2001:983:eeb4:1:c0de:69ff:fe2f:5599) has joined #ceph
[0:34] <gwinger> how do you think we can place a redundant RBD/nfs so that it can be accessed bey different hosts
[0:35] <gwinger> Thiet is only my thought, that it can grow like the cluster
[0:35] <gwinger> and accessible by all the hosts
[0:35] <SamYaple> gwinger: there are a few ways to do it, not of them are really good in my opinion
[0:35] <gwinger> like a NAS
[0:35] <SamYaple> what yo uare asking for is cephfs (which is stable as of Jewel)
[0:35] <gwinger> ok
[0:36] <blizzow> SamYaple: What do you mean single RBD as opposed to pool?
[0:36] <gwinger> Yes,
[0:36] <SamYaple> blizzow: you have virtualmachinces running on libvirt accessing a single RBD in a pool. for example rbd/web-server01
[0:36] <SamYaple> you cant have multiple virtual machines access rbd/web-server01
[0:36] <gwinger> I think, that it would be useful to have an nfs redundant
[0:37] <SamYaple> gwinger: you can, in an active-passive manner
[0:37] <gwinger> with rsync pacemaker and so on?
[0:37] <SamYaple> pacemaker, but no rsync
[0:37] <gwinger> or directly inside of a ceps cluster
[0:37] * KindOne (kindone@0001a7db.user.oftc.net) Quit (Quit: Hiring PHP developers does not contribute to the quota of employees with disabilities.)
[0:37] <SamYaple> rbd mounted on one host, it fails pacemaker mounts it on another and starts another nfs
[0:38] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) Quit (Ping timeout: 480 seconds)
[0:38] <blizzow> SamYaple: Not to be pedantic, but wouldn't that be referred to as an Image? The chain goes RBD -> Pool -> Image or FS.
[0:38] <gwinger> ok, with synced data on it?
[0:38] <SamYaple> blizzow: no
[0:38] <gwinger> should be for secondary cloudstack storage
[0:38] <SamYaple> gwinger: you dont need the data in multiple places since you can mount the rbd to another place
[0:39] <gwinger> ok, is the rbd replicated inside of the ceps cluster?
[0:39] <SamYaple> gwinger: correct
[0:39] <gwinger> Thanks, some things become clearer to me ;-)
[0:40] * jclm (~jclm@rrcs-69-193-93-163.nys.biz.rr.com) Quit (Quit: Leaving.)
[0:41] <gwinger> So I could decide to use a 33% of the size of the --pool secondary (the one for the SATA's) and let ceps replicate it to the 3 hosts
[0:41] <gwinger> Or can I use the complete pool for 1 rbd?
[0:42] <SamYaple> gwinger: well a pool has no size. the cluster has a size and any rbd in any pool could consume the entire clusters size.
[0:42] <gwinger> I have 12 4 TB SATA disks on 3 Hosts
[0:42] <SamYaple> the answer to what you are asking is "yes" though, you cna have one rbd the size of the cluster
[0:44] * KindOne (kindone@h118.58.16.72.dynamic.ip.windstream.net) has joined #ceph
[0:44] <SamYaple> blizzow: for the record here, RBD and Image are pretty much interchangable. RBD is probably the most common way to refer to it. an FS may go on the RBD.
[0:44] <SamYaple> So Pool -> RBD | Image -> FS
[0:47] <gwinger> so if I place a red or an image on the SATA pool, via crush map only useable on the sata, the I could tell the rib not to contain more than the size of the SATA'S and be sure, that the secondary pool will never grow larger than the size of the complete SATA Storage
[0:47] <gwinger> I this right?
[0:48] <gwinger> Asked in other words
[0:49] <gwinger> is the size of an rbd fixed? Or can it expand
[0:49] <SamYaple> no gwinger you can actually create an RBD _larger_ than the size of the cluster. so you have to control that
[0:50] <gwinger> yes, thats what I think
[0:50] <SamYaple> if you have 10TB of useable space on your OSDs that the secondary pool is mapped to, you want to make sure that you do not use more than 8TB (so you have 20% free)
[0:50] <SamYaple> filling a cluster full is bad all around
[0:51] <gwinger> Of course
[0:51] <SamYaple> you can expand an RBD. you can even do it live without unmounting it, but you have to be careful about it
[0:51] <gwinger> I have mapped all the OSD's of the STA disks to the pool named secondary
[0:52] <gwinger> this is together 48 TB
[0:52] <gwinger> crush map builds the mon tree that way
[0:53] <gwinger> So If i give the size for the rbd perhaps 47 TB then it can't grow bigger in worst case
[0:54] <gwinger> And I only have to map the rbd and mount it in active passive way to the 3 hosts
[0:54] <gwinger> Right
[0:54] <gwinger> ?
[0:55] * poller (~LRWerewol@7V7AAF7Y1.tor-irc.dnsbl.oftc.net) Quit ()
[0:57] <gwinger> Yet unclear: Is the data inside of a rib replicated by ceph? Or is the rbd as self replicated
[0:57] <gwinger> Sorry, I'm not long dealing with the material
[1:00] <SamYaple> gwinger: you set the max size of the RBD when you create it. it can't grow larger than that until you expand it with another command
[1:00] <SamYaple> the RBD itself is replicated, it is unaware of the filesystem
[1:00] * cathode (~cathode@50.232.215.114) Quit (Quit: Leaving)
[1:02] <gwinger> so I would have to calculate the size of the rbd that way, that it can be replicated to 3 Hosts in my case, If want redundancy and self healing via ceph. Then it is a perfect solution to do it like that
[1:03] <gwinger> Each host has 16 TB Space on Sata
[1:03] <gwinger> so I use 80% for the rbd
[1:03] <gwinger> 14TB around that
[1:04] <gwinger> I know what to do now ;-)
[1:04] <gwinger> Much thanx to you guys
[1:05] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) Quit (Ping timeout: 480 seconds)
[1:10] * MentalRay (~MentalRay@office-mtl1-nat-146-218-70-69.gtcomm.net) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[1:13] <EthanL> Anyone here a ceph setup expert?
[1:14] * vbellur (~vijay@71.234.224.255) Quit (Ping timeout: 480 seconds)
[1:14] * gwinger (~gwinger@ip5f5be42d.dynamic.kabel-deutschland.de) Quit (Quit: Verlassend)
[1:14] * vbellur (~vijay@173-13-111-22-NewEngland.hfc.comcastbusiness.net) has joined #ceph
[1:18] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[1:22] * Vacuum_ (~Vacuum@88.130.195.140) has joined #ceph
[1:22] * vata (~vata@cable-192.222.249.207.electronicbox.net) has joined #ceph
[1:25] * jacoo (~mLegion@freedom.ip-eend.nl) has joined #ceph
[1:27] <EthanL> SamYaple: Are there any known problems with this page: http://docs.ceph.com/docs/jewel/install/manual-deployment/
[1:27] <EthanL> ?
[1:27] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) Quit (Quit: valeech)
[1:28] * Vacuum__ (~Vacuum@88.130.196.150) Quit (Ping timeout: 480 seconds)
[1:29] <Kvisle> upgrading my hammer-osds to jewel is pretty disruptive
[1:30] <badone> EthanL: yes
[1:30] <badone> EthanL: if this is f24 or soemthing the ceph user will have no shell
[1:31] <EthanL> badone: f24?
[1:31] <SamYaple> EthanL: the docs are mostly in a state of kinda of working. what issue do you have?
[1:31] <badone> EthanL: Fedora 24
[1:32] <EthanL> badone, SamYaple, I am using debian Jessie and trying to set up a cluster with just a few nodes to play around with but I cant seem to get the mon working using these instructions
[1:32] <badone> EthanL: can you list what you've done and the results you got on a paste somewhere?
[1:33] <EthanL> I have tried ceph-deploy but that one complains as well but also I don't like how you have to set the permissions for it so I would prefer to just use ceph
[1:33] * Penn (~penn11.li@183.37.239.175) has joined #ceph
[1:34] <EthanL> badone I can would you like me to enter in all the commands I have been doing and then paste that entire section? or just the portion that is throwing the error?
[1:34] <badone> EthanL: let's start with the error
[1:36] * vbellur1 (~vijay@c-24-62-127-188.hsd1.ma.comcast.net) has joined #ceph
[1:37] <EthanL> http://pastebin.com/SH5duDDB
[1:37] <EthanL> I did change the ceph start mon.node1 to ceph3
[1:38] * reed (~reed@m9f0536d0.tmodns.net) Quit (Ping timeout: 480 seconds)
[1:38] * vbellur (~vijay@173-13-111-22-NewEngland.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[1:39] * rendar (~I@host218-193-dynamic.181-80-r.retail.telecomitalia.it) Quit (Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!)
[1:39] <badone> EthanL: what does /etc/ceph/ceph.client.admin.keyring have in it?
[1:40] <EthanL> [client.admin]
[1:40] <EthanL> key = AQA8/F5XDWf5CxAAwA46s0zHeYrV511SeXyK3Q==
[1:40] <EthanL> auid = 0
[1:40] <EthanL> caps mds = "allow"
[1:40] <EthanL> caps mon = "allow *"
[1:40] <EthanL> caps osd = "allow *"
[1:40] * xarses (~xarses@ip-64-134-222-177.public.wayport.net) Quit (Ping timeout: 480 seconds)
[1:41] <badone> EthanL: is that on ceph3 ?
[1:41] <EthanL> yes that is on ceph3
[1:41] <badone> EthanL: very odd given this error, unable to find a keyring on /etc/ceph/ceph.client.admin.keyring
[1:42] <EthanL> badone: that is exactly what is so confusing it seems like everything is there but it complains about not having the keyring files
[1:43] <badone> EthanL: can you run something like this?
[1:43] <badone> strace -eopen ceph osd lspools
[1:43] <badone> that should show you what files it is trying to open and whether the open returns an error
[1:44] * ChrisHolcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) has joined #ceph
[1:44] <EthanL> badone should it be spitting out several hundred lines?
[1:45] <badone> EthanL: yep, you'll need to look for the one with /etc/ceph/ceph.client.admin.keyring
[1:45] * fsimonce (~simon@host107-37-dynamic.251-95-r.retail.telecomitalia.it) Quit (Quit: Coyote finally caught me)
[1:49] <EthanL> badone according to a search with vim keyring only appears on the unable to find line
[1:49] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[1:49] <EthanL> badone as does /etc/ceph/
[1:50] <badone> EthanL: oops, try strace -f
[1:51] <badone> strace -f -eopen ./ceph osd lspools 2>&1|grep keyring
[1:51] <badone> strace -f -eopen ceph osd lspools 2>&1|grep keyring
[1:53] <EthanL> badone: http://pastebin.com/5UJARkUS
[1:53] <badone> Permission denied
[1:55] * jacoo (~mLegion@4MJAAGIME.tor-irc.dnsbl.oftc.net) Quit ()
[1:55] * KapiteinKoffie (~dontron@orion.enn.lu) has joined #ceph
[1:56] <EthanL> Okay, although it is saying that /etc/ceph/keyring does not exists but I cant see anything about that on lines 8,9,10 in the instuctions when it is talking about keyring
[1:57] <badone> EthanL: ceph will search for various files to find a keyring. It expects to not to be able some... but not all
[1:57] <EthanL> ah so it really is that I was not running ceph osd lspools as sudo
[1:58] <EthanL> in this case should I change the file permissions? or just use sudo
[1:59] * ronrib (~boswortr@45.32.242.135) has joined #ceph
[1:59] <badone> EthanL: normally, you'd use sudo if you aren't alrady root
[2:00] <EthanL> badone: thanks for your help sorry it ended up being so trivial of a problem and thanks for informing me about strace that should help in future problems I have
[2:00] * vbellur1 (~vijay@c-24-62-127-188.hsd1.ma.comcast.net) Quit (Ping timeout: 480 seconds)
[2:00] <EthanL> okay good to know I have been trying to stay away from sudo as from what I understand as far as security it is best to not use sudo but if it is required I will use it
[2:00] <badone> EthanL: hmmm, yes, that is a problem with that page isn't it?
[2:01] <badone> EthanL: np, glad to help. Sorry I didn't see the whole picture quicker :)
[2:01] * vbellur (~vijay@71.234.224.255) has joined #ceph
[2:01] <EthanL> That is okay I have been pounding away at this for a few days you got it much faster than I did lol
[2:01] <badone> EthanL: it will be gone soon since we are removing the shell for the ceph user
[2:01] <badone> the daemons will run as ceph but that user will have no login
[2:02] <EthanL> badone if it is okay I have a few more conceptual questions about ceph and what it can do out of the box and if I have to write anything for it
[2:02] * haomaiwang (~haomaiwan@li1068-35.members.linode.com) has joined #ceph
[2:02] <EthanL> badone Ah hmm okay so setup will be different soon? compared to what it is right now?
[2:02] <badone> EthanL: ask away, I'm not the most ceph knowledgeable guy but others can jump in hopefully
[2:02] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[2:03] <badone> EthanL: yes
[2:03] <badone> EthanL: not to mention a move from ceph-deploy to ceph-ansible based solutions
[2:03] <EthanL> badone: thanks I am wondering if it is possible to have a machine use its only storage for ceph?
[2:03] <badone> EthanL: if you are deploying a new cluster right now I'd give ceph-ansible a try
[2:04] <badone> EthanL: it's only storage?
[2:04] <badone> you mean a single disk?
[2:04] * gahan (gahan@breath.hu) Quit (Quit: leaving)
[2:05] <EthanL> Ie I have 4 (I could get more if needed) systems to try with right now and each one only has 1 hdd in it. I would like to make it so I can mount block devices as bootable disks over the network to each one
[2:05] <EthanL> this would allow me to move and save configurations of OS installs for bare metal like you can do for vm's
[2:06] <EthanL> so it would just be used as storage as in the storage would only be split up into mountable drives/blocks
[2:06] <badone> EthanL: the only way to do that would be directories, rather than disks for the OSD storage...
[2:06] <badone> EthanL: hoghly recommended to use a disk for each osd
[2:06] <badone> *highly*
[2:07] <EthanL> when you say directories what are you meaning like chroot? or directories inside of the disks?
[2:08] <badone> EthanL: ceph normally works by taking an entire disk for each osd (although journals can share an osd)
[2:08] <badone> it can work with partitions as well
[2:09] <badone> in testing it can use directories but that's not recommended for production IIRC
[2:09] <EthanL> when you say osd do you mean a single hdd? or do you mean the ceph cluster?
[2:09] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) has joined #ceph
[2:10] <EthanL> sorry I still don't quite understand all the parts and how they interact
[2:10] <badone> EthanL: you'll need to
[2:10] * haomaiwang (~haomaiwan@li1068-35.members.linode.com) Quit (Ping timeout: 480 seconds)
[2:11] <EthanL> I do know that I need to but considering the trouble I have had just getting the mon working I just seeing how feasible my idea is
[2:11] <EthanL> basically I want to do for physical machines what we can do with vm's
[2:11] <ronrib> ok i'm having a weird apt issue, i just upgraded one of my ceph nodes and it basically uninstalled ceph, now i'm seeing "ceph-common : Depends: librbd1 (= 10.2.1-1xenial) but 10.2.2-1xenial is to be installed"
[2:11] <EthanL> badone: am I making any sense?
[2:11] <badone> EthanL: have a read of this first, http://docs.ceph.com/docs/master/architecture/
[2:12] <ronrib> sure enough, "sudo apt-cache show librbd1" shows "Version: 10.2.2-1xenial"
[2:12] <badone> EthanL: you mean like Ironic?
[2:13] <EthanL> Okay I will read that It looks like it might be more indepth than the one on youtube that I found by Ross Turk
[2:14] <badone> EthanL: take a look at Ironic as well
[2:14] <badone> sounds like it might be what you are looking for
[2:15] <EthanL> I am thinking of intergrating with Ironic I don't need the rest of openstack or anything but I would like to be able to move installs around on partitions that are smaller than the physical disks in the machines ie there can be 10x the number of 'installs' compared to the number of drives
[2:15] <EthanL> I did look at ironc but I didn't see anything like that when I was looking through it I might have missed something
[2:15] <EthanL> *missed something though
[2:15] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[2:16] <badone> EthanL: not sure how a bare metal machine would boot from an rbd image, some tricks with krbd maybe? Don't know...
[2:17] <badone> EthanL: asking similar questions in Ironic's fora may help as well
[2:17] <EthanL> I figured I would have to make it a network drive and then do some fancy footwork to get the system to boot from from it
[2:17] <ronrib> ok i'm pretty sure the ubuntu 16.04 packages are broken, "apt-cache policy ceph" lists "Candidate: 10.2.2-1xenial" but "sudo apt-cache policy ceph-common" lists "Candidate: 10.2.1-1xenial"
[2:18] <EthanL> Okay I will hop on there, really the biggest thing is I need to be able to save OS installs and mount them later and to have more OS installs than the number of drives that I have
[2:20] * Concubidated (~cube@12.199.206.50) has joined #ceph
[2:21] <badone> EthanL: rbd images may work for that but you'll need to do some hoemwork to be sure... a directory based storage set-up might also prove a challenge without good working knowledge
[2:21] <badone> EthanL: although... let me check something
[2:22] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[2:23] * wushudoin (~wushudoin@38.99.12.237) Quit (Ping timeout: 480 seconds)
[2:24] <badone> EthanL: go take a look at ceph-ansible, it has an option "osd_directory" which will help you a lot I think
[2:24] <badone> there are a couple of introductory videos which will help you get started
[2:24] <badone> :q
[2:25] * KapiteinKoffie (~dontron@7V7AAF716.tor-irc.dnsbl.oftc.net) Quit ()
[2:25] * Xerati (~SquallSee@anonymous6.sec.nl) has joined #ceph
[2:27] <EthanL> Okay I will go look and see if it is what I am looking for Thanks!
[2:27] <EthanL> And If I understand it correctly the usual use case of ceph is just for storage of stuff?
[2:27] <EthanL> ie videos, or data etc?
[2:28] <badone> np, it should ease the pain of the install at least
[2:29] <badone> EthanL: can be used for block storage, VM images, etc and object storage and filesystem storage via cephfs..
[2:31] <EthanL> the filesystem storage is just like a large drive where everyone gets their own folders right? and block storage turns each hard drive into a block?
[2:37] * blizzow (~jburns@50.243.148.102) Quit (Remote host closed the connection)
[2:40] <EthanL> Another way to explain it is I would like to be able to take and make several luns and present those as iSCSI devices
[2:41] * huangjun (~kvirc@113.57.168.154) has joined #ceph
[2:41] * Penn (~penn11.li@183.37.239.175) Quit (Ping timeout: 480 seconds)
[2:41] <badone> EthanL: filesystem storage is just a filesystem like nfs but it is spread over all the drives, the block storage is block devices also spread over all the drives
[2:42] <EthanL> Based on what I am finding it looks like it can be done although I will have to do some extra leg work
[2:43] <EthanL> Thanks for all the info badone!
[2:43] <badone> yw
[2:46] * Steki (~steki@cable-89-216-193-98.dynamic.sbb.rs) has joined #ceph
[2:53] * BManojlovic (~steki@cable-89-216-226-90.dynamic.sbb.rs) Quit (Ping timeout: 480 seconds)
[2:54] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[2:55] * Xerati (~SquallSee@06SAAD0PK.tor-irc.dnsbl.oftc.net) Quit ()
[2:55] * Ralth (~MatthewH1@7V7AAF74F.tor-irc.dnsbl.oftc.net) has joined #ceph
[3:07] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[3:08] * Cybertinus (~Cybertinu@cybertinus.customer.cloud.nl) Quit (Ping timeout: 480 seconds)
[3:11] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) has joined #ceph
[3:15] * haomaiwang (~haomaiwan@li1068-35.members.linode.com) has joined #ceph
[3:16] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[3:16] * Cybertinus (~Cybertinu@cybertinus.customer.cloud.nl) has joined #ceph
[3:16] * Penn (~penn11.li@183.37.239.175) has joined #ceph
[3:17] * Nicola-1_ (~Nicola-19@x4db30f9c.dyn.telefonica.de) has joined #ceph
[3:20] * NTTEC (~nttec@119.93.91.136) has joined #ceph
[3:20] * NTTEC (~nttec@119.93.91.136) Quit (Remote host closed the connection)
[3:20] * NTTEC (~nttec@119.93.91.136) has joined #ceph
[3:23] <NTTEC> anyone talking here today?
[3:23] * Nicola-1980 (~Nicola-19@x4e37c18e.dyn.telefonica.de) Quit (Ping timeout: 480 seconds)
[3:25] * Ralth (~MatthewH1@7V7AAF74F.tor-irc.dnsbl.oftc.net) Quit ()
[3:26] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[3:27] * haomaiwang (~haomaiwan@li1068-35.members.linode.com) Quit (Remote host closed the connection)
[3:27] * valeech (~valeech@ip-64-134-185-188.public.wayport.net) has joined #ceph
[3:30] * kefu (~kefu@183.193.187.151) has joined #ceph
[3:34] * kefu_ (~kefu@114.92.120.18) has joined #ceph
[3:38] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[3:38] * kefu (~kefu@183.193.187.151) Quit (Ping timeout: 480 seconds)
[3:42] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[3:42] * yanzheng (~zhyan@118.116.112.205) has joined #ceph
[3:43] * sebastian-w (~quassel@212.218.8.139) Quit (Remote host closed the connection)
[3:43] * sebastian-w (~quassel@212.218.8.139) has joined #ceph
[3:44] * overclk|afk is now known as overclk
[3:50] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[3:50] * kefu_ (~kefu@114.92.120.18) Quit (Max SendQ exceeded)
[3:51] * kefu (~kefu@114.92.120.18) has joined #ceph
[3:52] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) Quit (Read error: Connection reset by peer)
[3:52] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) has joined #ceph
[3:55] * MatthewH12 (~datagutt@marylou.nos-oignons.net) has joined #ceph
[3:55] * shyu (~shyu@218.241.172.114) has joined #ceph
[3:57] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) Quit ()
[3:58] * Brochacho (~alberto@c-73-45-127-198.hsd1.il.comcast.net) Quit (Quit: Brochacho)
[3:58] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[4:06] * theTrav (~theTrav@ipc032.ipc.telstra.net) has joined #ceph
[4:07] <theTrav> so, has anyone experimented with wide and thin ceph deployments?
[4:08] <theTrav> eg, an openstack cluster where ceph is installed on compute nodes?
[4:08] <theTrav> the aim being that the cluster can be elastically scaled (add/remove servers without config updates)
[4:08] <theTrav> or at least minimal config updates and minimal downtime
[4:09] <theTrav> even better zero downtime
[4:11] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) Quit (Quit: Leaving)
[4:14] * sage (~quassel@2607:f298:6050:709d:2c96:8ea6:6d05:6512) Quit (Remote host closed the connection)
[4:14] * huangjun (~kvirc@113.57.168.154) Quit (Read error: Connection reset by peer)
[4:15] * sage (~quassel@2607:f298:6050:709d:25f0:f760:4bf0:473d) has joined #ceph
[4:15] * ChanServ sets mode +o sage
[4:16] * huangjun (~kvirc@113.57.168.154) has joined #ceph
[4:18] * chiluk (~quassel@172.34.213.162.lcy-01.canonistack.canonical.com) Quit (Quit: No Ping reply in 180 seconds.)
[4:20] * vanham (~vanham@191.185.29.65) has joined #ceph
[4:20] * chiluk (~quassel@172.34.213.162.lcy-01.canonistack.canonical.com) has joined #ceph
[4:22] * Penn (~penn11.li@183.37.239.175) Quit ()
[4:25] * MatthewH12 (~datagutt@4MJAAGISH.tor-irc.dnsbl.oftc.net) Quit ()
[4:25] * cheese^1 (~Throlkim@tor-exit0-readme.dfri.se) has joined #ceph
[4:25] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[4:28] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[4:29] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[4:30] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[4:31] * EinstCrazy (~EinstCraz@58.247.117.134) has joined #ceph
[4:33] * EinstCrazy (~EinstCraz@58.247.117.134) Quit (Remote host closed the connection)
[4:34] * vicente (~~vicente@125.227.238.55) has joined #ceph
[4:34] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) Quit (Ping timeout: 480 seconds)
[4:35] * EinstCrazy (~EinstCraz@58.247.117.134) has joined #ceph
[4:36] * EinstCra_ (~EinstCraz@58.247.117.134) has joined #ceph
[4:36] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) has joined #ceph
[4:38] * vanham (~vanham@191.185.29.65) has joined #ceph
[4:40] * theTrav (~theTrav@ipc032.ipc.telstra.net) Quit (Remote host closed the connection)
[4:42] * EinstCra_ (~EinstCraz@58.247.117.134) Quit (Remote host closed the connection)
[4:43] * EinstCra_ (~EinstCraz@58.247.117.134) has joined #ceph
[4:43] * EinstCrazy (~EinstCraz@58.247.117.134) Quit (Ping timeout: 480 seconds)
[4:45] * werm_ (~anonymous@vanquish.cacr.caltech.edu) has joined #ceph
[4:48] * werm_ is now known as terminalecho
[4:50] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[4:51] * IvanJobs (~ivanjobs@103.50.11.146) has joined #ceph
[4:51] <IvanJobs> hi cephers, I didn't see ceph-kvstore-tool in v0.94.6 hammer, why?
[4:53] * vanham (~vanham@191.185.29.65) has joined #ceph
[4:55] * cheese^1 (~Throlkim@7V7AAF77R.tor-irc.dnsbl.oftc.net) Quit ()
[4:55] * skney1 (~offer@chomsky.torservers.net) has joined #ceph
[4:56] * chandanc (~chandanc@116.197.184.13) has joined #ceph
[4:57] * lj (~liujun@111.202.176.44) Quit (Ping timeout: 480 seconds)
[5:04] <badone> IvanJobs: In what package set?
[5:04] <badone> IvanJobs: it seems to be listed in the rpm spec
[5:05] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) has joined #ceph
[5:06] <IvanJobs> badone: I just ssh to my ceph node trying "ceph " tab to see all available commands prefixed with ceph. But I didn't see ceph-kvstore-tool.
[5:07] <IvanJobs> maybe I didn't install this tool? can u provide the rpm spec file link for me ?
[5:07] <badone> IvanJobs: what OS?
[5:07] <IvanJobs> ubuntu 14.04
[5:07] <badone> is there a ceph-ttols package?
[5:07] <badone> *ceph-tools*
[5:08] <badone> it ships in a ceph-tools package in rpm land :)
[5:09] <IvanJobs> package ceph-tools is not installed
[5:10] <IvanJobs> I used ceph-deploy to install this ceph cluster, maybe ceph-deploy didn't install ceph-tools for me.
[5:11] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) has joined #ceph
[5:11] <badone> IvanJobs: check what packages you don't have compared to this list, http://download.ceph.com/debian-hammer/pool/main/c/ceph/
[5:11] <badone> might be ceph-test... not sure
[5:12] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[5:13] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[5:13] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) Quit (Ping timeout: 480 seconds)
[5:13] <badone> IvanJobs: yep, looks that way, https://pkgs.org/ubuntu-15.10/ubuntu-updates-universe-amd64/ceph-test_0.94.6-0ubuntu0.15.10.1_amd64.deb.html
[5:19] <IvanJobs> badone, thx, fixed my confusion.
[5:20] <badone> np
[5:25] * skney1 (~offer@06SAAD0U7.tor-irc.dnsbl.oftc.net) Quit ()
[5:25] * MonkeyJamboree (~ItsCrimin@hessel3.torservers.net) has joined #ceph
[5:30] * m8x` (~user@182.150.27.112) has joined #ceph
[5:30] * m8x (~user@182.150.27.112) Quit (Read error: Connection reset by peer)
[5:30] * theTrav (~theTrav@ipc032.ipc.telstra.net) has joined #ceph
[5:32] * m8x`` (~user@182.150.27.112) has joined #ceph
[5:33] * m8x` (~user@182.150.27.112) Quit (Read error: Connection reset by peer)
[5:33] * vanham (~vanham@191.185.29.65) has joined #ceph
[5:36] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[5:40] * deepthi (~deepthi@115.118.208.173) has joined #ceph
[5:40] * valeech (~valeech@ip-64-134-185-188.public.wayport.net) Quit (Quit: valeech)
[5:47] * EinstCra_ (~EinstCraz@58.247.117.134) Quit (Remote host closed the connection)
[5:47] * EinstCrazy (~EinstCraz@58.247.117.134) has joined #ceph
[5:51] * linuxkidd (~linuxkidd@ip70-189-207-54.lv.lv.cox.net) Quit (Quit: Leaving)
[5:52] * shylesh (~shylesh@45.124.227.70) has joined #ceph
[5:54] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[5:55] * MonkeyJamboree (~ItsCrimin@06SAAD0WO.tor-irc.dnsbl.oftc.net) Quit ()
[5:55] * Vidi (~Aal@64.ip-37-187-176.eu) has joined #ceph
[5:55] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[5:55] * EinstCrazy (~EinstCraz@58.247.117.134) Quit (Remote host closed the connection)
[5:57] * Vacuum__ (~Vacuum@i59F79AD0.versanet.de) has joined #ceph
[6:04] * Vacuum_ (~Vacuum@88.130.195.140) Quit (Ping timeout: 480 seconds)
[6:04] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[6:12] * IvanJobs_ (~ivanjobs@103.50.11.146) has joined #ceph
[6:12] * IvanJobs (~ivanjobs@103.50.11.146) Quit (Read error: Connection reset by peer)
[6:17] * IvanJobs (~ivanjobs@103.50.11.146) has joined #ceph
[6:17] * IvanJobs_ (~ivanjobs@103.50.11.146) Quit (Read error: Connection reset by peer)
[6:18] * rotbeard (~redbeard@2a02:908:df13:bb00:4464:93bf:4545:849e) has joined #ceph
[6:23] * mattbenjamin (~mbenjamin@76-206-42-50.lightspeed.livnmi.sbcglobal.net) has joined #ceph
[6:24] * shylesh (~shylesh@45.124.227.70) Quit (Remote host closed the connection)
[6:25] * Vidi (~Aal@06SAAD0XK.tor-irc.dnsbl.oftc.net) Quit ()
[6:25] * IvanJobs (~ivanjobs@103.50.11.146) Quit (Ping timeout: 480 seconds)
[6:28] * swami1 (~swami@49.44.57.241) has joined #ceph
[6:29] * IvanJobs (~ivanjobs@103.50.11.146) has joined #ceph
[6:30] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[6:31] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[6:33] * efirs (~firs@c-50-185-70-125.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[6:34] * rotbeard (~redbeard@2a02:908:df13:bb00:4464:93bf:4545:849e) Quit (Ping timeout: 480 seconds)
[6:34] * rotbeard (~redbeard@aftr-109-90-233-106.unity-media.net) has joined #ceph
[6:36] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Remote host closed the connection)
[6:36] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) Quit (Ping timeout: 480 seconds)
[6:38] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) has joined #ceph
[6:40] * mattbenjamin (~mbenjamin@76-206-42-50.lightspeed.livnmi.sbcglobal.net) Quit (Ping timeout: 480 seconds)
[6:41] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[6:42] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[6:43] * EinstCra_ (~EinstCraz@58.247.119.250) has joined #ceph
[6:44] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Remote host closed the connection)
[6:47] * swami1 (~swami@49.44.57.241) Quit (Ping timeout: 480 seconds)
[6:47] * swami1 (~swami@49.44.57.241) has joined #ceph
[6:51] * NTTEC_ (~nttec@203.177.235.23) has joined #ceph
[6:52] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[6:52] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[6:55] * eXeler0n (~kalleeen@tor-exit-node.gertrude.stuken.me) has joined #ceph
[6:56] * NTTEC (~nttec@119.93.91.136) Quit (Ping timeout: 480 seconds)
[6:57] * alexxy[home] (~alexxy@biod.pnpi.spb.ru) has joined #ceph
[6:57] * natarej (~natarej@101.188.54.14) has joined #ceph
[6:58] * miller7 (~user@ppp-94-64-50-84.home.otenet.gr) has joined #ceph
[7:01] * alexxy (~alexxy@144.206.128.196) Quit (Ping timeout: 480 seconds)
[7:02] * nils_ (~nils_@doomstreet.collins.kg) has joined #ceph
[7:06] * alexxy[home] (~alexxy@biod.pnpi.spb.ru) Quit (Ping timeout: 480 seconds)
[7:07] * vikhyat (~vumrao@121.244.87.116) has joined #ceph
[7:08] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[7:08] * alexxy (~alexxy@biod.pnpi.spb.ru) has joined #ceph
[7:11] * rdas (~rdas@121.244.87.116) has joined #ceph
[7:13] * sickology (~mio@vpn.bcs.hr) Quit (Read error: Connection reset by peer)
[7:14] * TomasCZ (~TomasCZ@yes.tenlab.net) Quit (Quit: Leaving)
[7:16] * kefu (~kefu@114.92.120.18) Quit (Max SendQ exceeded)
[7:17] * kefu (~kefu@114.92.120.18) has joined #ceph
[7:20] * sickology (~mio@vpn.bcs.hr) has joined #ceph
[7:21] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) has joined #ceph
[7:23] * vanham (~vanham@191.185.29.65) has joined #ceph
[7:25] * IvanJobs (~ivanjobs@103.50.11.146) Quit (Read error: Connection reset by peer)
[7:25] * eXeler0n (~kalleeen@06SAAD0ZZ.tor-irc.dnsbl.oftc.net) Quit ()
[7:25] * CobraKhan0071 (~Kottizen@7V7AAF8EB.tor-irc.dnsbl.oftc.net) has joined #ceph
[7:25] * IvanJobs (~ivanjobs@122.14.140.7) has joined #ceph
[7:26] * Georgyo (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) Quit (Quit: No Ping reply in 180 seconds.)
[7:27] * Georgyo (~georgyo@shamm.as) has joined #ceph
[7:30] <NTTEC_> anyone here tried to use calamari?
[7:33] * Georgyo_ (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) has joined #ceph
[7:36] * EinstCra_ (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[7:36] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[7:36] * Georgyo (~georgyo@shamm.as) Quit (Ping timeout: 480 seconds)
[7:37] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[7:40] * Georgyo (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) has joined #ceph
[7:41] * Georgyo_ (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) Quit (Ping timeout: 480 seconds)
[7:45] * T1 (~the_one@87.104.212.66) Quit (Read error: Connection reset by peer)
[7:48] * T1 (~the_one@87.104.212.66) has joined #ceph
[7:52] * aiicore (~aiicore@s30.linuxpl.com) Quit (Quit: leaving)
[7:52] * IvanJobs (~ivanjobs@122.14.140.7) Quit (Read error: Connection reset by peer)
[7:52] * aiicore (~aiicore@s30.linuxpl.com) has joined #ceph
[7:53] * IvanJobs (~ivanjobs@103.50.11.146) has joined #ceph
[7:53] * shylesh (~shylesh@121.244.87.118) has joined #ceph
[7:53] * theTrav (~theTrav@ipc032.ipc.telstra.net) Quit (Read error: Connection reset by peer)
[7:54] * theTrav (~theTrav@ipc032.ipc.telstra.net) has joined #ceph
[7:55] * CobraKhan0071 (~Kottizen@7V7AAF8EB.tor-irc.dnsbl.oftc.net) Quit ()
[7:55] * Kizzi (~Pettis@2.tor.exit.babylon.network) has joined #ceph
[7:56] * kawa2014 (~kawa@89.184.114.246) has joined #ceph
[7:56] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[8:02] <vikhyat> kefu: my query is
[8:02] <vikhyat> if we install fresh hammer cluster
[8:02] <kefu> iiuc, straw or straw2 is just user's call.
[8:02] <vikhyat> right
[8:03] * post-factum (~post-fact@vulcan.natalenko.name) Quit (Killed (NickServ (Too many failed password attempts.)))
[8:03] <vikhyat> but with optimal tunables we support straw2
[8:03] <vikhyat> correct
[8:03] * post-factum (~post-fact@2001:19f0:6c00:8846:5400:ff:fe0c:dfa0) has joined #ceph
[8:03] * karnan (~karnan@121.244.87.117) has joined #ceph
[8:03] <vikhyat> and if some one installing a fresh cluster we still get *Straw* not *straw2*
[8:03] <kefu> right. per sage, it has better locality and produce less data migration when osd is flapping.
[8:03] * vata (~vata@cable-192.222.249.207.electronicbox.net) Quit (Quit: Leaving.)
[8:04] <vikhyat> right for straw2
[8:04] <kefu> vikhyat: oh.
[8:05] <vikhyat> kefu: right
[8:05] <kefu> so, you suggest we have straw2 as the default algo in hammer?
[8:05] <vikhyat> right
[8:05] <kefu> we should*
[8:05] <vikhyat> if we install fresh cluster
[8:05] <vikhyat> like crushmap is getting created first time
[8:05] <vikhyat> or in Jewel ?
[8:05] <kefu> maybe it's for backward compatibility.
[8:05] <vikhyat> if it is not possible in hammer
[8:05] <kefu> as the client does not understand this.
[8:06] <vikhyat> kefu: but if some one is creating fresh cluster
[8:06] <vikhyat> I hope he will have client same or above
[8:06] <kefu> vikhyat, i am not sure.
[8:06] <kefu> imaging we are deploying an app
[8:06] <vikhyat> right got it
[8:06] <kefu> over the google play.
[8:06] <vikhyat> backward compatibility is very much important
[8:07] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[8:07] <vikhyat> hmm
[8:07] <kefu> it's on the wild. we can not (or have limited) control of it after the deployment.
[8:07] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) has joined #ceph
[8:07] * alexxy (~alexxy@biod.pnpi.spb.ru) Quit (Quit: No Ping reply in 180 seconds.)
[8:08] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) has joined #ceph
[8:09] <kefu> vikhyat, maybe we will have a librados in pure javascript one day. then we won't worry about this anymore.
[8:09] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) has joined #ceph
[8:09] * alexxy (~alexxy@biod.pnpi.spb.ru) has joined #ceph
[8:09] <vikhyat> kefu: :D
[8:09] <kefu> but it's but my understanding.
[8:10] <vikhyat> you are 100% right buddy mostly it is for backward compatibility
[8:10] <kefu> it's a good q though: when shall we change to straw2 as the default algo.
[8:10] <vikhyat> but going forward I think in Jewel or luminous we should make straw2 default
[8:10] <kefu> probably it's just a matter of time.
[8:10] <vikhyat> right
[8:10] <kefu> right.
[8:10] <vikhyat> so when luminous will release
[8:11] <vikhyat> we will have at max supported version LTS would be hammer
[8:11] <vikhyat> and then hammer does support straw2
[8:11] * IvanJobs (~ivanjobs@103.50.11.146) Quit (Remote host closed the connection)
[8:11] <vikhyat> hammer, jewel and luminous
[8:12] <vikhyat> all will support straw2
[8:12] <kefu> kraken.
[8:12] <vikhyat> LTS :D
[8:12] <kefu> oh, right.
[8:12] <vikhyat> kefu: thank you for all your inputs
[8:13] <vikhyat> kefu: now let me explain you motivation of my RFE
[8:13] <vikhyat> suppose user is running with optimal tunable in hammer
[8:13] <vikhyat> means he has "allowed_bucket_algs": 54, which means straw2 can be supported
[8:14] <vikhyat> with this cluster
[8:14] <vikhyat> but his crushmap is still in *straw2*
[8:14] <vikhyat> but without downloading crushmap or running $ceph osd crush dump
[8:15] <vikhyat> it is not possible to check if it is *straw* or *Straw2* inside the crushmap
[8:15] <theTrav> so how big can a ceph cluster get and still be somewhat sane?
[8:15] <vikhyat> so I was think if we can have a field in $ceph osd crush show-tunables which can tell the status of straw2 inside crushmap
[8:16] <theTrav> like, can I install it on 20 small-ish servers?
[8:16] <theTrav> with 5TB each?
[8:17] <kefu> vikhyat, i think "tunables" is more about what are allowed.
[8:17] <kefu> instead what it is now.
[8:17] <vikhyat> kefu: ahh okay
[8:17] <vikhyat> and for checking what is now
[8:17] <kefu> like what knobs do we have.
[8:18] <vikhyat> we need to run crush dump
[8:18] <vikhyat> hmm
[8:18] <kefu> right
[8:18] <vikhyat> got your point
[8:18] <vikhyat> kefu: thank you
[8:18] <vikhyat> !m kefu
[8:18] <kefu> vikhyat yw
[8:20] * saintpablo (~saintpabl@5.186.78.30) has joined #ceph
[8:21] <kefu> theTrav, there were some interesting discussion on the high density machines for deploying ceph. see http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-September/004184.html
[8:21] * gauravbafna (~gauravbaf@49.32.0.223) has joined #ceph
[8:22] * karnan (~karnan@121.244.87.117) Quit (Ping timeout: 480 seconds)
[8:22] * onyb (~ani07nov@119.82.105.66) has joined #ceph
[8:22] <kefu> but vikhyat, i agree, 54 is not very user friendly =)
[8:22] <vikhyat> kefu: right :D
[8:22] <vikhyat> kefu: if you want I can create a RFE
[8:23] <vikhyat> kefu: but same goes to crushmap also
[8:23] <kefu> vikhyat sure, better off create it over the tracker. so ppl might want to grab it as a low hanging fruit.
[8:23] <vikhyat> and I think field is integer not a string
[8:23] <vikhyat> kefu: ack
[8:24] <vikhyat> kefu: will do it may be I look into the code I will be able to do it will do it
[8:24] <kefu> vikhyat yeah. we can make it a string or an array, as long as it does not break any test suite.
[8:24] * theTrav (~theTrav@ipc032.ipc.telstra.net) Quit (Quit: Leaving...)
[8:24] <vikhyat> kefu: ack thanks for your guidance
[8:24] <kefu> there is some trade off though, 54 is why easier to test =D
[8:24] <kefu> s/why/way/
[8:25] <vikhyat> true :)
[8:25] * Kizzi (~Pettis@06SAAD02B.tor-irc.dnsbl.oftc.net) Quit ()
[8:25] * swami2 (~swami@49.32.0.224) has joined #ceph
[8:25] <vikhyat> I think it is used at multiple places so modifying many test suits could be ?
[8:25] * Miouge (~Miouge@91.177.17.5) has joined #ceph
[8:26] <kefu> vikhyat yeah. maybe it's an easy but non-trivial task.
[8:26] <vikhyat> kefu: true
[8:28] * swami1 (~swami@49.44.57.241) Quit (Ping timeout: 480 seconds)
[8:29] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) Quit (Ping timeout: 480 seconds)
[8:29] * Catsceo (~djidis__@orion.enn.lu) has joined #ceph
[8:31] * ade (~abradshaw@dslb-092-078-134-220.092.078.pools.vodafone-ip.de) has joined #ceph
[8:32] * karnan (~karnan@121.244.87.117) has joined #ceph
[8:39] * IvanJobs (~ivanjobs@103.50.11.146) has joined #ceph
[8:39] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[8:43] * saintpablos (~saintpabl@5.186.78.30) has joined #ceph
[8:45] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) Quit (Ping timeout: 480 seconds)
[8:48] * saintpablo (~saintpabl@5.186.78.30) Quit (Ping timeout: 480 seconds)
[8:50] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[8:50] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[8:51] * saintpablo (~saintpabl@5.186.78.30) has joined #ceph
[8:52] * saintpablos (~saintpabl@5.186.78.30) Quit (Ping timeout: 480 seconds)
[8:52] <NTTEC_> anyone here tried to use calamari?
[8:52] * saintpablo (~saintpabl@5.186.78.30) Quit (Read error: Connection reset by peer)
[8:55] * krypto (~krypto@sessfw99-sesbfw99-90.ericsson.net) Quit (Read error: Connection reset by peer)
[8:57] * saintpablo (~saintpabl@5.186.78.30) has joined #ceph
[8:57] * EinstCra_ (~EinstCraz@58.247.119.250) has joined #ceph
[8:58] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Ping timeout: 480 seconds)
[8:58] * dgurtner (~dgurtner@178.197.236.226) has joined #ceph
[8:58] * getup (~getup@gw.office.cyso.net) has joined #ceph
[8:59] * Catsceo (~djidis__@4MJAAGI34.tor-irc.dnsbl.oftc.net) Quit ()
[9:01] * getup (~getup@gw.office.cyso.net) Quit ()
[9:02] * getup (~getup@gw.office.cyso.net) has joined #ceph
[9:04] * derjohn_mob (~aj@p578b6aa1.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[9:05] * saintpablo (~saintpabl@5.186.78.30) Quit (Ping timeout: 480 seconds)
[9:12] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[9:13] <NTTEC_> I'm using calamari for my ceph and end up with this problem
[9:13] <NTTEC_> Ceph servers are connected to Calamari, but no Ceph cluster has been created yet. Please use ceph-deploy to create a cluster; please see the Inktank Ceph Enterprise documentation for more details.
[9:13] * EinstCra_ (~EinstCraz@58.247.119.250) Quit (Read error: No route to host)
[9:14] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[9:15] * Hemanth (~hkumar_@121.244.87.117) has joined #ceph
[9:16] * ngoswami (~ngoswami@121.244.87.116) has joined #ceph
[9:16] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[9:17] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[9:19] * davidz1 (~davidz@2605:e000:1313:8003:1df5:30c6:5bf4:e7c0) Quit (Quit: Leaving.)
[9:19] * sxc731 (~sxc731@xdsl-188-154-67-61.adslplus.ch) has joined #ceph
[9:19] * sxc731 (~sxc731@xdsl-188-154-67-61.adslplus.ch) Quit (Remote host closed the connection)
[9:21] * vanham (~vanham@191.185.29.65) has joined #ceph
[9:22] * chiluk (~quassel@172.34.213.162.lcy-01.canonistack.canonical.com) Quit (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.)
[9:24] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[9:24] * analbeard (~shw@support.memset.com) has joined #ceph
[9:28] * ade (~abradshaw@dslb-092-078-134-220.092.078.pools.vodafone-ip.de) Quit (Quit: Too sexy for his shirt)
[9:29] * derjohn_mob (~aj@fw.gkh-setu.de) has joined #ceph
[9:29] * fsimonce (~simon@host107-37-dynamic.251-95-r.retail.telecomitalia.it) has joined #ceph
[9:29] * Xeon06 (~anadrom@tor-exit.insane.us.to) has joined #ceph
[9:31] * allaok (~allaok@machine107.orange-labs.com) has joined #ceph
[9:33] * wbg (~wbg@ave.zedat.fu-berlin.de) has joined #ceph
[9:36] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[9:36] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[9:37] * rotbeard (~redbeard@aftr-109-90-233-106.unity-media.net) Quit (Quit: Leaving)
[9:38] * rraja (~rraja@121.244.87.117) has joined #ceph
[9:38] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[9:39] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[9:41] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) has joined #ceph
[9:44] <Walex> so I have just noticed that on several Ceph servers I have 17 OSDs and each has on average around 4,000 (four thousand) threads.
[9:44] <Walex> also on those Ceph servers several OSDs processes have accumulated many hours of system CPU, and usually system CPU time is about the same as user CPU time for OSDs.
[9:44] * getup (~getup@gw.office.cyso.net) Quit (Quit: Textual IRC Client: www.textualapp.com)
[9:48] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) has joined #ceph
[9:49] <nils_> Walex, can we see your ceph config?
[9:49] <nils_> Walex, half of the time in kernel seems also a bit weird.
[9:51] <nils_> 400-500 per OSD for me (Infernalis)
[9:52] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[9:52] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[9:52] * ade (~abradshaw@dslb-092-078-134-220.092.078.pools.vodafone-ip.de) has joined #ceph
[9:55] * vanham (~vanham@191.185.29.65) has joined #ceph
[9:58] * thomnico (~thomnico@2a01:e35:8b41:120:c090:921f:dbb7:2e1a) has joined #ceph
[9:59] * Xeon06 (~anadrom@4MJAAGI6P.tor-irc.dnsbl.oftc.net) Quit ()
[9:59] * brannmar (~ain@4MJAAGI75.tor-irc.dnsbl.oftc.net) has joined #ceph
[10:00] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Read error: No route to host)
[10:00] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[10:00] * NTTEC_ (~nttec@203.177.235.23) Quit (Remote host closed the connection)
[10:02] * tacticus (~tacticus@2400:8900::f03c:91ff:feae:5dcd) Quit (Ping timeout: 480 seconds)
[10:03] * vanham (~vanham@191.185.29.65) Quit (Ping timeout: 480 seconds)
[10:03] * bitserker (~toni@88.87.194.130) has joined #ceph
[10:04] * bara (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[10:05] * Steki (~steki@cable-89-216-193-98.dynamic.sbb.rs) Quit (Remote host closed the connection)
[10:06] * Steki (~steki@cable-89-216-193-98.dynamic.sbb.rs) has joined #ceph
[10:10] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) Quit (Ping timeout: 480 seconds)
[10:10] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) has joined #ceph
[10:19] * rendar (~I@host153-194-dynamic.18-79-r.retail.telecomitalia.it) has joined #ceph
[10:26] * thomnico (~thomnico@2a01:e35:8b41:120:c090:921f:dbb7:2e1a) Quit (Quit: Ex-Chat)
[10:27] * bliu (~liub@192.94.118.200) has joined #ceph
[10:28] * rotbeard (~redbeard@185.32.80.238) has joined #ceph
[10:29] * brannmar (~ain@4MJAAGI75.tor-irc.dnsbl.oftc.net) Quit ()
[10:30] * Spessu (~Tralin|Sl@politkovskaja.torservers.net) has joined #ceph
[10:32] * T1w (~jens@node3.survey-it.dk) has joined #ceph
[10:33] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Ping timeout: 480 seconds)
[10:35] * deepthi (~deepthi@115.118.208.173) Quit (Ping timeout: 480 seconds)
[10:43] * briner (~briner@129.194.16.54) Quit (Quit: briner)
[10:45] * briner (~briner@2001:620:600:1000:5d26:8eaa:97f0:8115) has joined #ceph
[10:47] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[10:48] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[10:49] * vanham (~vanham@191.185.29.65) has joined #ceph
[10:50] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[10:50] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[10:50] * linjan (~linjan@86.62.112.22) has joined #ceph
[10:50] * briner (~briner@2001:620:600:1000:5d26:8eaa:97f0:8115) Quit (Remote host closed the connection)
[10:51] * bliu (~liub@192.94.118.200) Quit (Quit: Leaving)
[10:54] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[10:59] * Spessu (~Tralin|Sl@4MJAAGI9I.tor-irc.dnsbl.oftc.net) Quit ()
[10:59] * Miho1 (~Jyron@193.90.12.87) has joined #ceph
[11:00] * onyb (~ani07nov@119.82.105.66) Quit (Quit: raise SystemExit())
[11:13] * owasserm (~owasserm@2001:984:d3f7:1:5ec5:d4ff:fee0:f6dc) has joined #ceph
[11:17] * yanzheng1 (~zhyan@118.116.112.205) has joined #ceph
[11:18] * yanzheng (~zhyan@118.116.112.205) Quit (Ping timeout: 480 seconds)
[11:23] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[11:23] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:27] * i_m (~ivan.miro@deibp9eh1--blueice2n6.emea.ibm.com) has joined #ceph
[11:29] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:89eb:703b:475a:a29) has joined #ceph
[11:29] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Read error: No route to host)
[11:29] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:29] * Miho1 (~Jyron@06SAAD09P.tor-irc.dnsbl.oftc.net) Quit ()
[11:31] <Walex> nils_: ah yes, thanks, I'll find it.
[11:31] <Walex> nils_: it is indeed quite weird. On another similar system I also see 300-500 threads per OSD, but even that is crazy
[11:31] * Walex had been bothered by users in the meantime
[11:32] <nils_> terrible, I really only want to run test clusters from now on
[11:33] <Walex> nils_: that would awesome here too :-)
[11:36] * swami2 (~swami@49.32.0.224) Quit (Ping timeout: 480 seconds)
[11:37] * topro (~prousa@host-62-245-142-50.customer.m-online.net) Quit (Quit: Konversation terminated!)
[11:38] * TMM (~hp@185.5.121.201) has joined #ceph
[11:38] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[11:38] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:39] * topro (~prousa@host-62-245-142-50.customer.m-online.net) has joined #ceph
[11:40] * topro (~prousa@host-62-245-142-50.customer.m-online.net) Quit ()
[11:40] * swami1 (~swami@49.32.0.224) has joined #ceph
[11:43] * topro (~prousa@host-62-245-142-50.customer.m-online.net) has joined #ceph
[11:45] * rakeshgm (~rakesh@121.244.87.117) has joined #ceph
[11:46] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Ping timeout: 480 seconds)
[11:52] * kefu is now known as kefu|afk
[11:52] * kefu|afk (~kefu@114.92.120.18) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[11:53] * Walex been distracted again
[11:54] * saintpablo (~saintpabl@gw01.mhitp.dk) has joined #ceph
[11:56] <Walex> nils_: in theb meantime 'htop' output on one of the affected systems: http://paste.ubuntu.com/17354867/
[11:56] * thomnico (~thomnico@2a01:e35:8b41:120:5d61:8693:c4d5:bd3) has joined #ceph
[11:57] <Walex> nils_: I have noticed that 'htop' itself has accumulated a lot of system time, but then it is reading > 65000 thread entries from the kernel
[11:57] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:57] * tumeric (~jcastro@89.152.250.115) has joined #ceph
[11:57] <tumeric> whois Be-El
[11:58] <tumeric> oops :p
[11:58] <tumeric> Be-El, Hi
[11:58] <tumeric> Hi guys
[11:58] <nils_> Walex, yeah, even on 40 cores that's quite a lot
[11:58] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[11:58] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[11:59] * bara_ (~bara@213.175.37.12) has joined #ceph
[11:59] <Walex> nils_: and this is the entire 'ceph.conf', in practice all defaults http://paste.ubuntu.com/17354915/
[11:59] <Kvisle> can one do --convert-filestore on an osd before one attempts to bring it online?
[11:59] <tumeric> I have a doubt, I created CephFS, two pools for cephfs with the same PG Num. Turns out only one pool is getting filled, and due to to this I have the same space available for the pool thats getting full and the other one thats empty
[11:59] * swami2 (~swami@49.44.57.241) has joined #ceph
[11:59] <tumeric> a_data 5 43629k 0 19873G 1167237
[11:59] <tumeric> a_metadata 6 1509G 6.99 19873G 27666616
[11:59] <Walex> nils_: the Ceph cluster is not very loaded
[12:00] <tumeric> Is it possible to change (increase space available on a_metadata) without having to delete the pools?
[12:00] * PuyoDead (~notmyname@tor-exit.eecs.umich.edu) has joined #ceph
[12:00] <tumeric> (dont mind the names)
[12:00] <nils_> Walex, everything else is default?
[12:00] <Walex> nils_: yes, AFAIK
[12:00] <nils_> Walex, since it's of course possible to inject arguments via ceph tell
[12:00] <Walex> nils_: it was a pretty much standard 'ceph-deploy' run. I didn't do it.
[12:00] <nils_> oh yeah I never used that.
[12:01] * hybrid512 (~walid@195.200.189.206) has joined #ceph
[12:02] <Walex> nils_: http://paste.ubuntu.com/17354968/
[12:02] * onyb (~ani07nov@119.82.105.66) has joined #ceph
[12:03] * swami1 (~swami@49.32.0.224) Quit (Ping timeout: 480 seconds)
[12:04] * bara (~bara@nat-pool-brq-t.redhat.com) Quit (Ping timeout: 482 seconds)
[12:04] <Walex> it is a very strange situation, I suspect some system issue more than Ceph, as 'gmetad' (Ganglia stats aggregator) also consumes a lot of system time.
[12:04] <nils_> Walex, you could use strace or perf to collect which syscalls the process is issuing.
[12:04] <Walex> but I can't imagine what Ceph can do with > 4,000 threads per OSD on a mostly queiscent system
[12:05] <Walex> nils_: yes, been thinkign of that, and also kernel profiling
[12:07] <nils_> Walex, I'd also check if htop is even right about the amount of threads by checking /proc/pidofcephosd/status (Threads)
[12:09] * allaok (~allaok@machine107.orange-labs.com) has left #ceph
[12:11] * miller7 (~user@ppp-94-64-50-84.home.otenet.gr) Quit (Ping timeout: 480 seconds)
[12:11] <Walex> nils_: yes, that matches: http://paste.ubuntu.com/17355204/
[12:14] <Walex> nils_: 10 seconds of kernel counters: http://paste.ubuntu.com/17355238/ (as in http://www.pixelbeat.org/programming/profiling/)
[12:16] * rdas (~rdas@121.244.87.116) has joined #ceph
[12:24] * miller7 (~user@2a02:587:2500:1f00:c870:648d:a170:b15e) has joined #ceph
[12:28] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) Quit (Ping timeout: 480 seconds)
[12:29] * PuyoDead (~notmyname@4MJAAGJDP.tor-irc.dnsbl.oftc.net) Quit ()
[12:30] * dgurtner (~dgurtner@178.197.236.226) Quit (Read error: Connection reset by peer)
[12:31] * chiluk (~quassel@172.34.213.162.lcy-01.canonistack.canonical.com) has joined #ceph
[12:36] * saintpablo (~saintpabl@gw01.mhitp.dk) Quit (Ping timeout: 480 seconds)
[12:39] * NTTEC (~nttec@122.53.162.158) has joined #ceph
[12:39] <NTTEC> Ceph servers are connected to Calamari, but no Ceph cluster has been created yet. Please use ceph-deploy to create a cluster; please see the Inktank Ceph Enterprise documentation for more details.
[12:39] <NTTEC> I'm using calamari for my ceph and end up with this problem
[12:50] * LongyanG (~long@15255.s.time4vps.eu) Quit (Remote host closed the connection)
[12:56] * tumeric is now known as turmeric
[12:56] * LongyanG (~long@15255.s.time4vps.eu) has joined #ceph
[12:59] * rajatg (~rajat@49.32.0.180) has joined #ceph
[12:59] * Chaos_Llama (~Helleshin@4MJAAGJF5.tor-irc.dnsbl.oftc.net) has joined #ceph
[13:00] * rajatg (~rajat@49.32.0.180) Quit ()
[13:03] * chandanc (~chandanc@116.197.184.13) Quit (Ping timeout: 480 seconds)
[13:04] * vicente (~~vicente@125.227.238.55) Quit (Quit: Leaving)
[13:06] * chengpeng (~chris@180.168.170.2) Quit (Ping timeout: 480 seconds)
[13:07] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Read error: Connection reset by peer)
[13:07] * Hemanth (~hkumar_@121.244.87.117) Quit (Ping timeout: 480 seconds)
[13:07] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[13:08] * rakeshgm (~rakesh@121.244.87.117) Quit (Ping timeout: 480 seconds)
[13:11] * NTTEC (~nttec@122.53.162.158) Quit (Remote host closed the connection)
[13:12] * b0e (~aledermue@213.95.25.82) has joined #ceph
[13:14] * karnan (~karnan@121.244.87.117) Quit (Ping timeout: 480 seconds)
[13:16] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:89eb:703b:475a:a29) Quit (Remote host closed the connection)
[13:17] * Hemanth (~hkumar_@121.244.87.118) has joined #ceph
[13:18] * rakeshgm (~rakesh@121.244.87.118) has joined #ceph
[13:20] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:19b7:2a8b:465f:7429) has joined #ceph
[13:24] * rakeshgm (~rakesh@121.244.87.118) Quit (Quit: Leaving)
[13:25] * rakeshgm (~rakesh@121.244.87.118) has joined #ceph
[13:25] <Tetard> is there another gui than calamari btw ? seems to me it's not actively maintained anymore
[13:27] <sep> Tetard, it is not ? i was looking for a gui to help my cli challenged coworkers, and calmari comes up in every search.
[13:29] * Chaos_Llama (~Helleshin@4MJAAGJF5.tor-irc.dnsbl.oftc.net) Quit ()
[13:29] * Wizeon (~darkid@anonymous6.sec.nl) has joined #ceph
[13:33] * bara_ (~bara@213.175.37.12) Quit (Ping timeout: 480 seconds)
[13:33] * huangjun (~kvirc@113.57.168.154) Quit (Ping timeout: 480 seconds)
[13:34] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) has joined #ceph
[13:39] * saintpablo (~saintpabl@gw01.mhitp.dk) has joined #ceph
[13:42] * Hemanth_ (~hkumar_@121.244.87.117) has joined #ceph
[13:43] * bara_ (~bara@nat-pool-brq-t.redhat.com) has joined #ceph
[13:44] * neurodrone (~neurodron@pool-100-35-225-168.nwrknj.fios.verizon.net) Quit (Quit: neurodrone)
[13:44] <turmeric> There are many, not sure if any of them is actually good
[13:45] <turmeric> inkscope, calamari, one from intel
[13:46] * IvanJobs_ (~ivanjobs@101.231.33.158) has joined #ceph
[13:46] * IvanJobs_ (~ivanjobs@101.231.33.158) Quit (Remote host closed the connection)
[13:48] * Hemanth (~hkumar_@121.244.87.118) Quit (Ping timeout: 480 seconds)
[13:48] * rakeshgm (~rakesh@121.244.87.118) Quit (Ping timeout: 480 seconds)
[13:49] * swami2 (~swami@49.44.57.241) Quit (Ping timeout: 480 seconds)
[13:51] * IvanJobs (~ivanjobs@103.50.11.146) Quit (Read error: Connection reset by peer)
[13:51] * sickology (~mio@vpn.bcs.hr) Quit (Read error: Connection reset by peer)
[13:53] * dgurtner (~dgurtner@178.197.236.226) has joined #ceph
[13:54] <Heebie> Is it possible to change a replicated pool to an erasure-encoded pool?
[13:56] * ira (~ira@c-24-34-255-34.hsd1.ma.comcast.net) has joined #ceph
[13:57] * sickology (~mio@vpn.bcs.hr) has joined #ceph
[13:57] * bniver (~bniver@nat-pool-bos-u.redhat.com) has joined #ceph
[13:57] * rakeshgm (~rakesh@121.244.87.117) has joined #ceph
[13:57] * dgurtner (~dgurtner@178.197.236.226) Quit ()
[13:58] * dgurtner (~dgurtner@178.197.236.226) has joined #ceph
[13:59] * Wizeon (~darkid@4MJAAGJHB.tor-irc.dnsbl.oftc.net) Quit ()
[14:01] * joelc (~joelc@cpe-24-28-78-20.austin.res.rr.com) has joined #ceph
[14:03] * swami1 (~swami@49.32.0.240) has joined #ceph
[14:04] * Nephyrin (~PuyoDead@tor-exit-4.all.de) has joined #ceph
[14:07] * ChrisHolcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) Quit (Ping timeout: 480 seconds)
[14:16] * garphy is now known as garphy`aw
[14:16] * Nephyrin (~PuyoDead@7V7AAF8TE.tor-irc.dnsbl.oftc.net) Quit (Remote host closed the connection)
[14:16] * Bj_o_rn (~Grum@79.134.255.200) has joined #ceph
[14:17] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) has joined #ceph
[14:19] <sebastian-w> Tetard: There's something in the works, called openATTIC
[14:22] * skyrat (~skyrat@178.17.3.251) has joined #ceph
[14:25] <Tetard> ack - for some reason I thought I remembered another one, but couldn't remember the name
[14:26] <Tetard> openATTIC seems like a complete storage solution
[14:27] * rotbeard (~redbeard@185.32.80.238) Quit (Quit: Leaving)
[14:28] <skyrat> Hi, what is the recommended way of restarting daemons in the cluster on Debian Jessie? The doc says using upstart, but this doesn't apply for Debian jessie. another page sayis use systemv init script, but this just finishes with success. I was only able to use systemctl restart on the node. Is there a way of restarting remotely from admin node? On admin node: `sudo /etc/init.d/ceph -a restart osd` just finishes, no output, no action. `sudo /etc/init.d/ceph -a
[14:28] <skyrat> restart osd.3` throws error: /etc/init.d/ceph: osd.3 not found (/etc/ceph/ceph.conf defines , /var/lib/ceph defines ) what does it mean? Can someone help?
[14:36] * huangjun (~kvirc@117.152.68.154) has joined #ceph
[14:37] * tlaxkit (~hexchat@84.120.66.20.dyn.user.ono.com) has joined #ceph
[14:37] * huangjun|2 (~kvirc@117.151.52.53) has joined #ceph
[14:37] * garphy`aw is now known as garphy
[14:39] * sickology (~mio@vpn.bcs.hr) Quit (Read error: Connection reset by peer)
[14:40] * swami1 (~swami@49.32.0.240) Quit (Quit: Leaving.)
[14:40] * gauravbafna (~gauravbaf@49.32.0.223) Quit (Remote host closed the connection)
[14:40] * sickology (~mio@vpn.bcs.hr) has joined #ceph
[14:41] <zdzichu> skyrat: systemctl restart ceph-osd@3
[14:41] <zdzichu> try it, it's how it works on fedora
[14:42] * i_m (~ivan.miro@deibp9eh1--blueice2n6.emea.ibm.com) Quit (Quit: Leaving.)
[14:44] * huangjun (~kvirc@117.152.68.154) Quit (Ping timeout: 480 seconds)
[14:44] * kefu (~kefu@183.193.187.151) has joined #ceph
[14:46] * Bj_o_rn (~Grum@06SAAD1G9.tor-irc.dnsbl.oftc.net) Quit ()
[14:46] * chrisinajar (~Inverness@tor-exit.eecs.umich.edu) has joined #ceph
[14:46] <skyrat> zdzichu, thanks, yes, I already mentioned this command is the only way I found that works, at least tries to restart the service
[14:47] * shaunm (~shaunm@74.83.215.100) Quit (Quit: Ex-Chat)
[14:48] <skyrat> additionally, the `systemctl restart ceph-osd@3` throws error that another ceph-osd is running. Yes, of course it is running, that's why I need to restart...
[14:49] * joelc (~joelc@cpe-24-28-78-20.austin.res.rr.com) Quit (Remote host closed the connection)
[14:52] * kefu_ (~kefu@114.92.120.18) has joined #ceph
[14:53] * georgem (~Adium@206.108.127.16) has joined #ceph
[14:54] * kefu (~kefu@183.193.187.151) Quit (Ping timeout: 480 seconds)
[14:55] * georgem (~Adium@206.108.127.16) Quit ()
[14:57] * vanham (~vanham@191.185.29.65) Quit (Remote host closed the connection)
[14:59] * rwheeler (~rwheeler@172.56.12.93) has joined #ceph
[15:01] * chengpeng (~chris@222.73.33.154) has joined #ceph
[15:01] <Heebie> skyrat: Did you end up using the same SSD journal partition for two different OSD's accidentally? (check in /var/lib/ceph/osd/osd.<NN> to see what the symlink is.) I did that before.. typo when using ceph-deploy.
[15:01] <Heebie> and those are the messages I received.
[15:02] * gregmark (~Adium@68.87.42.115) has joined #ceph
[15:02] * georgem (~Adium@206.108.127.16) has joined #ceph
[15:02] * ibravo (~ibravo@72.83.69.64) has joined #ceph
[15:06] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) Quit (Read error: Connection reset by peer)
[15:06] <skyrat> Heebie, no, the journal symlink points to different target device for all osds on the node
[15:07] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) has joined #ceph
[15:08] * deepthi (~deepthi@115.118.145.26) has joined #ceph
[15:10] * sxc731 (~sxc731@194.230.159.188) has joined #ceph
[15:10] <Heebie> skyrat: I thought mine did until I took a more careful look. ;)
[15:10] * lmb (~Lars@2a02:8109:8100:1d2c:4053:922a:2b98:1f74) has joined #ceph
[15:11] * huangjun|2 (~kvirc@117.151.52.53) Quit (Ping timeout: 480 seconds)
[15:15] <skyrat> Heebie, I got your point, but I checked carefully.
[15:16] * chrisinajar (~Inverness@4MJAAGJK2.tor-irc.dnsbl.oftc.net) Quit ()
[15:20] * dyasny (~dyasny@cable-192.222.152.136.electronicbox.net) has joined #ceph
[15:24] <Heebie> Sorry it wasn't a help, then.
[15:26] <wujson> hi gyus
[15:26] <wujson> i've compiled ceph without tcmalloc and with jemalloc - it still uses tcmalloc
[15:26] <wujson> should I do something else to use jemalloc instead of tcmalloc?
[15:28] <nils_> wujson, what's the configure line you used?
[15:29] <wujson> nils_: --without-tcmalloc --with jemalloc
[15:30] <wujson> nils_: root@bld-osd4:/usr/src/CephZGita/ceph/debian# grep without rules
[15:30] <wujson> extraopts += --without-tcmalloc --with-jemalloc extraopts += --without-libatomic-ops
[15:30] <wujson> i've added it into rules file, and it's being used by configure mechanism
[15:31] <nils_> wujson, I think I already once modified the rules file to use jemalloc, gotta check what I did on that box
[15:32] <wujson> nils_: it worked, it let me built deb packages, but.. it still uses tcmalloc :(
[15:32] <Heebie> I asked about converting a replicated pool to an erasure-encoded pool. I think this answer is "no" maybe even "HAHAHAHAHA ARE YOU KIDDING?!?!?" but I wanted to ask the question.
[15:32] * kefu_ (~kefu@114.92.120.18) Quit (Max SendQ exceeded)
[15:33] <nils_> wujson, can you pastebin the ldd output on the binary?
[15:33] * kefu (~kefu@114.92.120.18) has joined #ceph
[15:34] * bene (~bene@nat-pool-bos-t.redhat.com) has joined #ceph
[15:36] <Heebie> Is there some easy way to install Calimari? The only thing I see in ceph-deploy is adding a ceph node to Calimari.
[15:36] <Heebie> (or.. as per conversation earlier there's something better.. what's that?) :)
[15:37] * kefu (~kefu@114.92.120.18) Quit (Max SendQ exceeded)
[15:37] * post-factum (~post-fact@2001:19f0:6c00:8846:5400:ff:fe0c:dfa0) Quit (Killed (NickServ (Too many failed password attempts.)))
[15:37] * kefu (~kefu@114.92.120.18) has joined #ceph
[15:37] <wujson> nils_: sure
[15:37] * post-factum (~post-fact@vulcan.natalenko.name) has joined #ceph
[15:38] <wujson> nils_: https://paste.fedoraproject.org/379512/97902146/
[15:39] <nils_> wujson, that is rather weird, I assume you have the jemalloc dev packages installed?
[15:40] <wujson> nils_: of course, without them it's impossible to build --with-jemalloc
[15:40] <wujson> nils_: i'll check binary built from tarball
[15:40] <wujson> oh.. i've deleted it
[15:40] <nils_> wujson, I think I had a few issues building with --with-jemalloc
[15:42] <wujson> nils_: could you take a look at 'rules' file? and, to be sure - have you built deb packages or from vanila tarball?
[15:42] <nils_> deb packages
[15:42] <nils_> for xenial if memory serves
[15:42] <wujson> nils_: so your rules file could do the trick.. ;)
[15:42] <nils_> but the machine I built on isn't turned on at the moment
[15:43] * chandanc (~chandanc@106.51.24.115) has joined #ceph
[15:43] * yanzheng1 (~zhyan@118.116.112.205) Quit (Quit: This computer has gone to sleep)
[15:45] <wujson> nils_: any ideas what should I try?
[15:46] <nils_> wujson, not really at the moment, I'll check my rules file once I'm near that server again
[15:46] <wujson> nils_: it'd be great
[15:46] * Shadow386 (~ylmson@ns316491.ip-37-187-129.eu) has joined #ceph
[15:51] * oms101 (~oms101@pb-d-128-141-242-179.cern.ch) has joined #ceph
[15:54] <oms101> th
[15:55] <oms101> ops sorry wrong window
[15:58] * m0zes__ (~mozes@n117m02.cis.ksu.edu) has joined #ceph
[16:01] * T1w (~jens@node3.survey-it.dk) Quit (Ping timeout: 480 seconds)
[16:01] * ibravo2 (~ibravo@72.198.142.104) has joined #ceph
[16:02] * kefu (~kefu@114.92.120.18) Quit (Max SendQ exceeded)
[16:03] * kefu (~kefu@114.92.120.18) has joined #ceph
[16:03] * xarses (~xarses@c-73-202-191-48.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[16:06] * rdas (~rdas@121.244.87.116) Quit (Quit: Leaving)
[16:07] * squizzi (~squizzi@107.13.31.195) has joined #ceph
[16:07] * joelc (~joelc@cpe-24-28-78-20.austin.res.rr.com) has joined #ceph
[16:08] * saintpablo (~saintpabl@gw01.mhitp.dk) Quit (Ping timeout: 480 seconds)
[16:08] * ibravo (~ibravo@72.83.69.64) Quit (Ping timeout: 480 seconds)
[16:09] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[16:10] * antongribok (~antongrib@216.207.42.140) has joined #ceph
[16:11] * dgurtner (~dgurtner@178.197.236.226) Quit (Read error: No route to host)
[16:11] * bene2 (~bene@nat-pool-bos-t.redhat.com) has joined #ceph
[16:11] * cholcombe (~chris@c-73-180-29-35.hsd1.or.comcast.net) has joined #ceph
[16:12] * joelc (~joelc@cpe-24-28-78-20.austin.res.rr.com) Quit (Remote host closed the connection)
[16:12] * chandanc_ (~chandanc@106.51.24.50) has joined #ceph
[16:15] * bene (~bene@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[16:15] * chandanc__ (~chandanc@106.51.25.226) has joined #ceph
[16:16] * gwinger (~gwinger@ip5f5be42d.dynamic.kabel-deutschland.de) has joined #ceph
[16:16] * rraja (~rraja@121.244.87.117) Quit (Quit: Leaving)
[16:16] <gwinger> hit there
[16:16] * Shadow386 (~ylmson@4MJAAGJOV.tor-irc.dnsbl.oftc.net) Quit ()
[16:17] * oms101 (~oms101@pb-d-128-141-242-179.cern.ch) Quit (Ping timeout: 480 seconds)
[16:18] * chandanc (~chandanc@106.51.24.115) Quit (Ping timeout: 480 seconds)
[16:19] * chandanc (~chandanc@106.51.225.118) has joined #ceph
[16:19] * mattbenjamin (~mbenjamin@12.118.3.106) has joined #ceph
[16:20] * chandanc_ (~chandanc@106.51.24.50) Quit (Ping timeout: 480 seconds)
[16:23] * xarses (~xarses@ip-64-134-222-177.public.wayport.net) has joined #ceph
[16:24] * chandanc__ (~chandanc@106.51.25.226) Quit (Ping timeout: 480 seconds)
[16:26] * dneary (~dneary@pool-96-233-46-27.bstnma.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[16:27] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[16:31] * hybrid512 (~walid@195.200.189.206) Quit (Remote host closed the connection)
[16:31] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Remote host closed the connection)
[16:32] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[16:33] * b0e (~aledermue@213.95.25.82) Quit (Quit: Leaving.)
[16:39] * EthanL (~ldapclone@cce02cs4036-fa12-z.ams.hpecore.net) Quit (Ping timeout: 480 seconds)
[16:39] * vata (~vata@207.96.182.162) has joined #ceph
[16:40] * gwinger (~gwinger@ip5f5be42d.dynamic.kabel-deutschland.de) Quit (Quit: Verlassend)
[16:44] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Remote host closed the connection)
[16:44] * linjan (~linjan@86.62.112.22) Quit (Ping timeout: 480 seconds)
[16:45] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[16:46] * chrisinajar (~dontron@tor1e1.privacyfoundation.ch) has joined #ceph
[16:48] * gwinger (~gwinger@2a02:8108:42c0:a68:f1a0:b20c:b5d7:950a) has joined #ceph
[16:48] * rossdylan (~rossdylan@2605:6400:1:fed5:22:68c4:af80:cb6e) Quit (Ping timeout: 480 seconds)
[16:49] * gwinger_ (~gwinger@ip5f5be42d.dynamic.kabel-deutschland.de) has joined #ceph
[16:49] * gwinger (~gwinger@2a02:8108:42c0:a68:f1a0:b20c:b5d7:950a) has left #ceph
[16:49] * joshd1 (~jdurgin@71-92-201-212.dhcp.gldl.ca.charter.com) has joined #ceph
[16:51] * wushudoin (~wushudoin@2601:646:9501:d2b2:2ab2:bdff:fe0b:a6ee) has joined #ceph
[16:53] * danieagle (~Daniel@191.255.39.11) Quit (Quit: Obrigado por Tudo! :-) inte+ :-))
[16:53] * EthanL (~lamberet@cce02cs4036-fa12-z.ams.hpecore.net) has joined #ceph
[16:54] * vikhyat (~vumrao@121.244.87.116) Quit (Quit: Leaving)
[16:54] * ntpttr_laptop (~ntpttr@192.55.54.45) has joined #ceph
[16:56] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Remote host closed the connection)
[16:59] * cathode (~cathode@50.232.215.114) has joined #ceph
[17:00] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[17:02] * analbeard (~shw@support.memset.com) Quit (Quit: Leaving.)
[17:03] * swami1 (~swami@27.7.164.182) has joined #ceph
[17:05] * sxc731 (~sxc731@194.230.159.188) Quit (Ping timeout: 480 seconds)
[17:09] * deepthi (~deepthi@115.118.145.26) Quit (Ping timeout: 480 seconds)
[17:10] * debian112 (~bcolbert@24.126.201.64) has joined #ceph
[17:14] * dgurtner (~dgurtner@178.197.236.226) has joined #ceph
[17:16] * chrisinajar (~dontron@7V7AAF81N.tor-irc.dnsbl.oftc.net) Quit ()
[17:16] * kalmisto (~Cue@3.tor.exit.babylon.network) has joined #ceph
[17:18] <derjohn_mob> Hello, if I remove a node from a running ceph cluster my running VMs get stuck for until some timeout is reached (I use noout, because I just want to reboot the node)... Is there some way to tell ceph that "node XY / osd XY will go down" ?
[17:18] * Brochacho (~alberto@c-73-45-127-198.hsd1.il.comcast.net) has joined #ceph
[17:19] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Remote host closed the connection)
[17:19] * gwinger (~gwinger@ip5f5be42d.dynamic.kabel-deutschland.de) has joined #ceph
[17:19] * gwinger_ (~gwinger@ip5f5be42d.dynamic.kabel-deutschland.de) Quit (Read error: Connection reset by peer)
[17:25] * shylesh (~shylesh@121.244.87.118) Quit (Remote host closed the connection)
[17:27] * sxc731 (~sxc731@xdsl-188-154-67-61.adslplus.ch) has joined #ceph
[17:28] * xarses_ (~xarses@ip-64-134-222-177.public.wayport.net) has joined #ceph
[17:29] <turmeric> just stop the service or put it out of the cluster
[17:29] * xarses_ (~xarses@ip-64-134-222-177.public.wayport.net) Quit (Remote host closed the connection)
[17:29] * xarses_ (~xarses@ip-64-134-222-177.public.wayport.net) has joined #ceph
[17:30] <turmeric> ceph osd set out
[17:31] <turmeric> you can also set no recover no backfill to prevent recover
[17:33] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[17:33] * xarses (~xarses@ip-64-134-222-177.public.wayport.net) Quit (Ping timeout: 480 seconds)
[17:36] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Remote host closed the connection)
[17:37] * yanzheng (~zhyan@118.116.112.205) has joined #ceph
[17:37] * TMM (~hp@185.5.121.201) Quit (Quit: Ex-Chat)
[17:39] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[17:40] * chandanc_ (~chandanc@106.51.24.128) has joined #ceph
[17:40] * rwheeler (~rwheeler@172.56.12.93) Quit (Ping timeout: 480 seconds)
[17:42] <rkeene> derjohn_mob, Setting noout just means the OSD won't get ejected, things will still try to talk to it and hang until they can
[17:44] * yanzheng (~zhyan@118.116.112.205) Quit (Quit: This computer has gone to sleep)
[17:44] * chandanc (~chandanc@106.51.225.118) Quit (Ping timeout: 480 seconds)
[17:46] * kalmisto (~Cue@4MJAAGJTN.tor-irc.dnsbl.oftc.net) Quit ()
[17:47] * shaunm (~shaunm@74.83.215.100) has joined #ceph
[17:48] * garphy is now known as garphy`aw
[17:49] * yanzheng (~zhyan@118.116.112.205) has joined #ceph
[17:52] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Read error: Connection reset by peer)
[17:55] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[17:57] <MaZ-> has anyone used the multisite rgw stuff in jewel yet?
[17:58] <MaZ-> i somehow have a zonegroup which contains correct zone settings, but retrieving each zone shows no 'endpoints' setting, and attempting to commit the period reports "could not find connection for zone or zonegroup id"
[18:07] * IvanJobs_ (~ivanjobs@183.192.78.179) has joined #ceph
[18:07] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Read error: Connection reset by peer)
[18:07] * yanzheng (~zhyan@118.116.112.205) Quit (Quit: This computer has gone to sleep)
[18:08] * linuxkidd (~linuxkidd@ip70-189-207-54.lv.lv.cox.net) has joined #ceph
[18:09] * miller7 (~user@2a02:587:2500:1f00:c870:648d:a170:b15e) Quit (Ping timeout: 480 seconds)
[18:09] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[18:09] * chandanc_ (~chandanc@106.51.24.128) Quit (Ping timeout: 480 seconds)
[18:12] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[18:14] * Concubidated (~cube@12.199.206.50) Quit (Quit: Leaving.)
[18:14] * ade (~abradshaw@dslb-092-078-134-220.092.078.pools.vodafone-ip.de) Quit (Quit: Too sexy for his shirt)
[18:15] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) has joined #ceph
[18:15] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) Quit ()
[18:15] * tlaxkit (~hexchat@84.120.66.20.dyn.user.ono.com) Quit (Quit: ??Adi??s!)
[18:16] * Frostshifter1 (~Lite@7V7AAF86B.tor-irc.dnsbl.oftc.net) has joined #ceph
[18:17] * IvanJobs_ (~ivanjobs@183.192.78.179) Quit (Read error: Connection reset by peer)
[18:18] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[18:18] * thomnico (~thomnico@2a01:e35:8b41:120:5d61:8693:c4d5:bd3) Quit (Quit: Ex-Chat)
[18:18] * dgurtner (~dgurtner@178.197.236.226) Quit (Read error: Connection reset by peer)
[18:20] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[18:21] * vanham (~vanham@191.185.29.65) has joined #ceph
[18:25] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) has joined #ceph
[18:27] * sxc731 (~sxc731@xdsl-188-154-67-61.adslplus.ch) Quit (Quit: Leaving.)
[18:28] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Ping timeout: 480 seconds)
[18:28] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Read error: Connection reset by peer)
[18:29] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[18:30] * haomaiwang (~haomaiwan@li1068-35.members.linode.com) has joined #ceph
[18:30] * dgurtner (~dgurtner@178.197.236.226) has joined #ceph
[18:32] * Hemanth_ (~hkumar_@121.244.87.117) Quit (Ping timeout: 480 seconds)
[18:32] * rakeshgm (~rakesh@121.244.87.117) Quit (Ping timeout: 481 seconds)
[18:33] * onyb (~ani07nov@119.82.105.66) Quit (Quit: raise SystemExit())
[18:35] <MaZ-> lol, okay so it's because my endpoints were a subdomain of the root s3 bucket domain
[18:36] <MaZ-> instead of /admin/period hitting the admin civetweb handler, it was hitting the s3 one
[18:36] <MaZ-> and not doing anything O_o
[18:37] * Be-El (~blinke@nat-router.computational.bio.uni-giessen.de) Quit (Quit: Leaving.)
[18:39] * ngoswami (~ngoswami@121.244.87.116) Quit (Quit: Leaving)
[18:40] * scg (~zscg@valis.gnu.org) has joined #ceph
[18:40] * bara_ (~bara@nat-pool-brq-t.redhat.com) Quit (Quit: Bye guys! (??????????????????? ?????????)
[18:41] * vasu (~vasu@c-73-231-60-138.hsd1.ca.comcast.net) has joined #ceph
[18:42] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) has joined #ceph
[18:42] <- *johnavp1989* To prove that you are human, please enter the result of 8+3
[18:44] * yanzheng (~zhyan@118.116.112.205) has joined #ceph
[18:45] <Heebie> derjohn_mob: setting noout is how you do that, but if your number of replicas of available data goes below the minimum, it will still freeze I/O on the pool to protect the data.
[18:46] <Heebie> turmeric seems to have a good answer for that, though. :)
[18:46] * Frostshifter1 (~Lite@7V7AAF86B.tor-irc.dnsbl.oftc.net) Quit ()
[18:49] * yanzheng (~zhyan@118.116.112.205) Quit (Quit: This computer has gone to sleep)
[18:50] * ntpttr_laptop_ (~ntpttr@192.55.54.45) has joined #ceph
[18:50] * ntpttr_laptop (~ntpttr@192.55.54.45) Quit (Remote host closed the connection)
[18:52] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Remote host closed the connection)
[18:53] * vata (~vata@207.96.182.162) Quit (Remote host closed the connection)
[18:53] * swami1 (~swami@27.7.164.182) Quit (Quit: Leaving.)
[18:55] * valeech (~valeech@wsip-70-166-79-23.ga.at.cox.net) Quit (Read error: Connection reset by peer)
[18:55] * garphy`aw is now known as garphy
[18:57] * karnan (~karnan@106.51.128.216) has joined #ceph
[18:57] <derjohn_mob> Heebie, turmeric ? What is that good answer ?
[18:58] <turmeric> derjohn_mob, what's your purpose? you want to reboot a machine without the ceph recovering part?
[18:59] <antongribok> derjohn_mob, just curios, is the timeout you're seeing a few seconds, or something like 300 seconds (5 minutes)?
[18:59] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) has joined #ceph
[18:59] <derjohn_mob> turmeric, correct ! I just wanted to Upgrade RAM, so the node is down for some minutes and I want to live with being degraded but dont want services to be impacted.
[19:00] <T1> hopefully it should just be a few seconds freezee until a secondary copy of the blocks are provided
[19:00] * ntpttr_laptop_ (~ntpttr@192.55.54.45) Quit (Quit: Leaving)
[19:00] <derjohn_mob> antongribok, it was some minutes.
[19:00] <antongribok> I feel like something is misconfigured in your setup, rebooting with noout should not normally impact storage services in a healthy cluster
[19:00] <antongribok> yeah, I'd say something's definitely off
[19:00] <T1> +1
[19:01] <T1> I've rebooted nodes while maintaining 400MB/s throughput
[19:01] <turmeric> ceph osd set norecover and ceph osd set nobackfill
[19:01] <turmeric> it will prevent the cluster from recovering
[19:02] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[19:02] <turmeric> just make sure you are still able to serve IO without this OSD/host
[19:03] <antongribok> if you do "ceph osd set noout" there is no need to do norecover and nobackfill
[19:03] * kawa2014 (~kawa@89.184.114.246) Quit (Quit: Leaving)
[19:03] <antongribok> something else is wrong here
[19:03] <turmeric> yup
[19:03] <turmeric> same thing
[19:03] <derjohn_mob> Where should I look into ?
[19:04] <turmeric> derjohn_mob, what is the status of the cluster if you reboot?
[19:04] <turmeric> ceph -s
[19:04] <derjohn_mob> health HEALTH_OK
[19:04] * linjan (~linjan@176.193.195.109) has joined #ceph
[19:04] <turmeric> during the reboot?
[19:05] <antongribok> what's the output of "ceph osd dump |grep <pool_name>" (should be a single line)
[19:05] <turmeric> just do ceph df ceph osd pool ls detail
[19:05] <turmeric> and post it here
[19:05] <turmeric> ceph df and ceph osd pool ls detail are differnet commands
[19:05] <derjohn_mob> $ ceph osd dump |grep ssd
[19:05] <derjohn_mob> pool 7 'ssd' replicated size 2 min_size 1 crush_ruleset 1 object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 52142 owner 18446744073709551615 flags hashpspool min_read_recency_for_promote 1 stripe_width 0
[19:08] <derjohn_mob> turmeric, http://pastebin.com/raw/W4FZj3Qk
[19:08] * mykola (~Mikolaj@193.93.217.59) has joined #ceph
[19:08] <turmeric> ceph -s and ceph df please
[19:08] <derjohn_mob> I might note, that there was one mon with clock skew detected.
[19:08] <antongribok> interesting, looks like this is a cache tier for another pool... are the hosts that you're rebooting holding both spinning disks and SSDs, or do you have SSD-based hosts and spinning-drive based hosts in separate machines?
[19:08] <turmeric> derjohn_mob, sudo apt-get install ntp
[19:09] <derjohn_mob> sure, ntp is running on all hosts
[19:09] <antongribok> the clock skew could be normal if that MON was recently rebooted, but if not, that could be a problem
[19:09] <derjohn_mob> (btw, physcial hosts with nothing else than ceph ...)
[19:09] * swami1 (~swami@27.7.164.182) has joined #ceph
[19:10] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Ping timeout: 480 seconds)
[19:10] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[19:10] <derjohn_mob> turmeric, http://pastebin.com/raw/P6BfA6Fg
[19:11] * vata (~vata@207.96.182.162) has joined #ceph
[19:11] <turmeric> thanks, how are your osd divided? into how many hosts?
[19:11] <derjohn_mob> antongribok, the have the (legacy) default pool "rbd" with spinning disks and a seconds pool "ssd" (which we mainly use .... for e.g. backup space we use the rbd / spinning ones)
[19:12] <turmeric> the crush map is default?
[19:12] <derjohn_mob> ceph osd tree | grep host | wc -l
[19:12] <derjohn_mob> 7
[19:12] <turmeric> normally with your setup and with noout you should be able to still serve IO, depending on how many hosts you have and how pgs are divded
[19:13] <derjohn_mob> turmeric, we must edit the crush map to place the ssd disks in the ssd pool.
[19:13] <derjohn_mob> besides that it default ....
[19:15] <turmeric> with min_size =1 you can still serve IO with just one replica, so I think you will have no issues
[19:15] <turmeric> with the reboot
[19:16] <antongribok> yeah, I agree, not sure what's going on... strange
[19:16] <derjohn_mob> turmeric, where do I set the min_size=1 ?
[19:16] <turmeric> the pools have this set already
[19:16] <turmeric> you have more than one host, right?
[19:17] <derjohn_mob> serveral computes nodes als rbd clients, 7 ceph nodes as "storage pools", three of then mon
[19:17] <turmeric> if you have, I guess you wont be in trouble, but maybe you can wait for an answer from someone more experienced than me
[19:18] * MentalRay (~MentalRay@MTRLPQ42-1176054809.sdsl.bell.ca) has joined #ceph
[19:20] <derjohn_mob> thanks anyway turmeric ans antongribok !
[19:20] <turmeric> derjohn_mob, ceph is made to be redudant, it splits the replicas around so losing one node wont impact you
[19:20] <turmeric> except in some specific situations
[19:20] * saintpablo (~saintpabl@185.85.5.78) has joined #ceph
[19:20] * Da_Pineapple (~Kwen@dourneau.me) has joined #ceph
[19:22] <derjohn_mob> thx <3 <3 ... I will re-phrase the question later on again ... maybe someone else has an idea.
[19:22] <turmeric> np
[19:22] * dgurtner (~dgurtner@178.197.236.226) Quit (Read error: Connection reset by peer)
[19:22] <turmeric> good luck
[19:22] * sbfox (~Adium@vancouver.xmatters.com) has joined #ceph
[19:24] * turmeric (~jcastro@89.152.250.115) Quit (Quit: Leaving)
[19:24] * sbfox (~Adium@vancouver.xmatters.com) Quit ()
[19:25] <antongribok> yeah, for some reason when I saw min_read_recency_for_promote I thought you had cache tiering
[19:25] <antongribok> sorry dude, would be very curious what's the actual problem
[19:25] <antongribok> definitely feel like I'm missing something :(
[19:25] * lmb (~Lars@2a02:8109:8100:1d2c:4053:922a:2b98:1f74) Quit (Ping timeout: 480 seconds)
[19:26] * davidz (~davidz@2605:e000:1313:8003:e00c:3a22:cea9:dfd1) has joined #ceph
[19:26] * kefu (~kefu@114.92.120.18) Quit (Quit: My Mac has gone to sleep. ZZZzzz???)
[19:29] * saintpablo (~saintpabl@185.85.5.78) Quit (Ping timeout: 480 seconds)
[19:30] * BartVB (~bartvb@92-111-39-39.static.chello.nl) Quit (Ping timeout: 480 seconds)
[19:31] * dneary (~dneary@nat-pool-bos-u.redhat.com) has joined #ceph
[19:31] * sbfox (~Adium@vancouver.xmatters.com) has joined #ceph
[19:31] * derjohn_mob (~aj@fw.gkh-setu.de) Quit (Ping timeout: 480 seconds)
[19:31] * Georgyo (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) Quit (Quit: No Ping reply in 180 seconds.)
[19:33] * Georgyo (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) has joined #ceph
[19:36] * stiopa (~stiopa@cpc73832-dals21-2-0-cust453.20-2.cable.virginm.net) has joined #ceph
[19:39] * Georgyo_ (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) has joined #ceph
[19:41] * Georgyo (~georgyo@2600:3c03:e000:71:f03c:91ff:feae:505c) Quit (Ping timeout: 480 seconds)
[19:42] * Brochacho (~alberto@c-73-45-127-198.hsd1.il.comcast.net) Quit (Quit: Brochacho)
[19:42] * lmb (~Lars@ip5b41f0a4.dynamic.kabel-deutschland.de) has joined #ceph
[19:42] * gauravba_ (~gauravbaf@122.172.193.249) has joined #ceph
[19:42] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Read error: Connection reset by peer)
[19:43] * chandanc_ (~chandanc@116.197.184.12) has joined #ceph
[19:44] * Racpatel (~Racpatel@2601:641:200:4c30:4e34:88ff:fe87:9abf) Quit (Ping timeout: 480 seconds)
[19:45] * bitserker (~toni@88.87.194.130) Quit (Ping timeout: 480 seconds)
[19:45] * lcurtis (~lcurtis@47.19.105.250) has joined #ceph
[19:50] * gauravba_ (~gauravbaf@122.172.193.249) Quit (Ping timeout: 480 seconds)
[19:50] * Da_Pineapple (~Kwen@4MJAAGJZY.tor-irc.dnsbl.oftc.net) Quit ()
[19:55] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[19:57] * sigsegv (~sigsegv@188.26.140.132) has joined #ceph
[19:57] * Racpatel (~Racpatel@c-73-170-66-165.hsd1.ca.comcast.net) has joined #ceph
[20:00] * IvanJobs (~ivanjobs@183.192.78.179) has joined #ceph
[20:03] * mhackett (~mhack@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[20:04] * fdmanana (~fdmanana@2001:8a0:6e0c:6601:19b7:2a8b:465f:7429) Quit (Ping timeout: 480 seconds)
[20:08] * IvanJobs (~ivanjobs@183.192.78.179) Quit (Ping timeout: 480 seconds)
[20:10] * joshd1 (~jdurgin@71-92-201-212.dhcp.gldl.ca.charter.com) Quit (Quit: Leaving.)
[20:14] * mhackett (~mhack@nat-pool-bos-u.redhat.com) has joined #ceph
[20:16] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Remote host closed the connection)
[20:19] * m0zes__ (~mozes@n117m02.cis.ksu.edu) Quit (Ping timeout: 480 seconds)
[20:20] * efirs (~firs@c-50-185-70-125.hsd1.ca.comcast.net) has joined #ceph
[20:20] * kalleeen (~kiasyn@7V7AAF9CW.tor-irc.dnsbl.oftc.net) has joined #ceph
[20:22] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[20:24] * TomasCZ (~TomasCZ@yes.tenlab.net) has joined #ceph
[20:30] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Ping timeout: 480 seconds)
[20:30] * Hemanth_ (~hkumar_@103.228.221.135) has joined #ceph
[20:31] * mivaho (~quassel@2001:983:eeb4:1:c0de:69ff:fe2f:5599) Quit (Ping timeout: 480 seconds)
[20:37] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[20:40] * mhackett (~mhack@nat-pool-bos-u.redhat.com) Quit (Ping timeout: 480 seconds)
[20:45] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Ping timeout: 480 seconds)
[20:46] * swami1 (~swami@27.7.164.182) Quit (Quit: Leaving.)
[20:50] * mhackett (~mhack@nat-pool-bos-t.redhat.com) has joined #ceph
[20:50] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) Quit (Ping timeout: 480 seconds)
[20:50] * kalleeen (~kiasyn@7V7AAF9CW.tor-irc.dnsbl.oftc.net) Quit ()
[20:50] * luckz (~smf68@192.42.115.101) has joined #ceph
[21:05] * reed (~reed@172.56.39.34) has joined #ceph
[21:07] * reed (~reed@172.56.39.34) Quit ()
[21:07] * sbfox (~Adium@vancouver.xmatters.com) Quit (Quit: Leaving.)
[21:08] * sbfox (~Adium@vancouver.xmatters.com) has joined #ceph
[21:09] <TheSov2> is public network=X/X still a thing?
[21:15] * onyb (~ani07nov@119.82.105.66) has joined #ceph
[21:17] * masteroman (~ivan@93-142-220-86.adsl.net.t-com.hr) has joined #ceph
[21:20] * masteroman (~ivan@93-142-220-86.adsl.net.t-com.hr) Quit ()
[21:20] * luckz (~smf68@06SAAD124.tor-irc.dnsbl.oftc.net) Quit ()
[21:21] <terminalecho> Hi, Is anyone familiar with how crushs handle replication, is it per OSD, or node? does it replicate to OSDs on the same node? in the case of SSD journals does it know that the OSDs use the same journal device and treat it as a single entity?
[21:21] * datagutt (~arsenaali@37.48.81.27) has joined #ceph
[21:24] * chandanc_ (~chandanc@116.197.184.12) Quit (Ping timeout: 480 seconds)
[21:26] <T1> rtfm?
[21:27] <T1> it's quite extensivly explained
[21:27] <rkeene> terminalecho, The default CRUSH map uses the failure domain of a host
[21:27] * Miouge (~Miouge@91.177.17.5) Quit (Quit: Miouge)
[21:28] <terminalecho> ok, so the crush map is where to look, thanks
[21:31] * bene (~bene@nat-pool-bos-t.redhat.com) has joined #ceph
[21:31] <terminalecho> also, has anyone here tested high speed data transfers? in your opinion, which is best to use for transfers in and out of Ceph, SSD journals on the nodes, SSD cache tier, or both?
[21:33] * Miouge (~Miouge@91.177.17.5) has joined #ceph
[21:33] * oms101 (~oms101@22-198-28-81.ftth.cust.kwaoo.net) has joined #ceph
[21:34] * bene2 (~bene@nat-pool-bos-t.redhat.com) Quit (Ping timeout: 480 seconds)
[21:38] * Hemanth_ (~hkumar_@103.228.221.135) Quit (Quit: Leaving)
[21:44] * TiCPU (~owrt@2001:470:1c:40::2) Quit (Read error: Connection reset by peer)
[21:44] * TiCPU (~owrt@c216.218.54-96.clta.globetrotter.net) has joined #ceph
[21:46] * bene (~bene@nat-pool-bos-t.redhat.com) Quit (Quit: Konversation terminated!)
[21:48] <TheSov2> is there any way to ask ceph what the cluster network is
[21:48] <TheSov2> i need to verify
[21:49] <antongribok> terminalecho, it really depends on your Ceph hardware config and your storage workload
[21:49] <antongribok> in some cases a cache tier can potentially be faster, but if you have spinning disks, and do any kind of write workload, you probably want to have SSD-based journals first
[21:50] <georgem> TheSov2: dump the running config on the OSD
[21:50] * datagutt (~arsenaali@06SAAD14A.tor-irc.dnsbl.oftc.net) Quit ()
[21:50] <terminalecho> ideally the idea is to read 2Gb-10Gb files as fast as possible, but also transfer them in and out of Ceph as fast as possible.
[21:50] * Tralin|Sleep (~Xerati@tor-exit-01.1d4.us) has joined #ceph
[21:51] <TheSov2> georgem, and i do that how?
[21:52] * Miouge (~Miouge@91.177.17.5) Quit (Quit: Miouge)
[21:52] <georgem> TheSov2: ceph daemon /var/run/ceph/ceph-osd.104.asok config show | grep public
[21:52] <terminalecho> I have some machines with 12Gb SAS disks and 40Gb interfaces, 2 high endurance SSDs and 10 enterprise spinners in each
[21:54] <TheSov2> ok works good
[21:55] <SamYaple> terminalecho: so is the idea to use the 2 ssds and have a 5:1 hdd:ssd ratio?
[21:57] * krypto (~krypto@G68-90-105-165.sbcis.sbc.com) has joined #ceph
[21:57] <terminalecho> right, this is what i was thinking, I was wondering how best to configure the journals, like maybe 200GB per spinner(1TB SSD). Does this look optimal, or is it overkill?
[21:58] <SamYaple> terminalecho: well you want it as large as possible to prevent ssd wear which can be a problem on some types of ssds
[21:58] * onyb (~ani07nov@119.82.105.66) Quit (Quit: raise SystemExit())
[21:58] * rwheeler (~rwheeler@12.218.204.2) has joined #ceph
[21:58] <SamYaple> but what I actually do is use bcache. ssd is the cache dev, hdd is the backing dvices
[21:58] <SamYaple> it doesnt limit the failure domain (lose ssd still lose other drives) but it give way better performance
[21:58] <antongribok> SSD wear and garbage collection
[21:59] <antongribok> so, +1 for bigger SSDs :)
[21:59] <SamYaple> neither of those are issues when using bcache btw. because the whole drive is used
[21:59] <antongribok> SamYaple, quick question for you on bcache, how do you deploy Ceph, do you use ceph-ansible?
[22:00] <SamYaple> antongribok: ive used ceph-deploy, ceph-ansible, and a homegrown ceph I build in docker containers (it is implemented in openstack/kolla)
[22:00] <antongribok> do you use file-based OSDs, or do you setup partitions on top of the bcache devices?
[22:02] <SamYaple> with bcache you setup a cache device and 'n' number of backing devices
[22:02] <SamYaple> each blockdevice used as a backing device generates a new bcache dev (/dev/bcache0, /dev/bcache1, etc)
[22:02] <antongribok> correct
[22:02] <SamYaple> so im confused as to your question?
[22:02] <antongribok> I was just curious how Ceph was sitting on top of those /dev/bcache0... devices
[22:03] <SamYaple> do you mean file based _journals_?
[22:03] <antongribok> yeah, sorry, forgot that terminology
[22:03] * mykola (~Mikolaj@193.93.217.59) Quit (Quit: away)
[22:03] <SamYaple> gotcha. yea i used file based journals at one point. now i do partitions
[22:04] <antongribok> oh, interesting
[22:04] <SamYaple> had to patch bcache for that https://yaple.net/2016/03/31/bcache-partitions-and-dkms/
[22:04] <SamYaple> working on pushing it upstream
[22:04] <antongribok> any particular reason for not using file-based journals?
[22:04] <SamYaple> file based journals introduces more writes
[22:05] <SamYaple> journal writes to file, and the the file system journal _also_ must write it out
[22:05] * karnan (~karnan@106.51.128.216) Quit (Remote host closed the connection)
[22:05] <SamYaple> so 4x writes, instead of 3x writes
[22:05] <SamYaple> 1 ceph journal, 2 file system journal, 3 file system file
[22:05] <zdzichu> SamYaple: why not create separate bcache device for each partition?
[22:05] <zdzichu> you could select different caching modes for journal and data
[22:06] <SamYaple> zdzichu: i could, but then i would be partitioning the underlying disk. why do that?
[22:06] <zdzichu> erm, because you partitioning bcache device anyway?
[22:06] <SamYaple> zdzichu: there is no benefit to that. there is stil la single failure domain. may as well take full advantage of writeback everywhere
[22:06] <SamYaple> so you mean the backing device, or the bcache device?
[22:07] <SamYaple> on the _bcache_ device i have /dev/bcache0p1 /dev/bcache0p2 for journal and data respectively
[22:07] <SamYaple> but on the backing device (/dev/sda) I have a single partition
[22:11] <antongribok> SamYaple, yeah, that's a good point on the extra write...
[22:11] <antongribok> I'm planning on testing bcache and lvmcache (dm-cache) in the next few weeks with Ceph to see how they compare in performance
[22:11] * xarses_ (~xarses@ip-64-134-222-177.public.wayport.net) Quit (Ping timeout: 480 seconds)
[22:12] * lx0 (~aoliva@lxo.user.oftc.net) has joined #ceph
[22:13] <antongribok> it kind of sucks that you can't really do true write-back caching with lvmcache currently, although supposedly DM developers are working on this
[22:15] <antongribok> BTW, I've seen your blog post previously while researching bcache :)
[22:18] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[22:20] * Tralin|Sleep (~Xerati@7V7AAF9GT.tor-irc.dnsbl.oftc.net) Quit ()
[22:23] * xarses_ (~xarses@64.124.158.100) has joined #ceph
[22:25] * rendar (~I@host153-194-dynamic.18-79-r.retail.telecomitalia.it) Quit (Ping timeout: 480 seconds)
[22:27] * georgem (~Adium@206.108.127.16) Quit (Ping timeout: 480 seconds)
[22:30] <SamYaple> antongribok: big issue i have with dm-cache is the need to partition a cache area _per_ backing device
[22:30] <SamYaple> bcache allow dynamically adding/removing backing devices
[22:34] <antongribok> yeah, that's a very good point
[22:34] * sigsegv (~sigsegv@188.26.140.132) has left #ceph
[22:34] * johnavp1989 (~jpetrini@pool-100-14-10-2.phlapa.fios.verizon.net) Quit (Ping timeout: 480 seconds)
[22:34] <antongribok> on the other hand, I like that you can take an existing live logical volume and enable a cache for it on the fly
[22:38] * gauravbafna (~gauravbaf@122.172.193.249) has joined #ceph
[22:40] * wjw-freebsd (~wjw@176.74.240.9) has joined #ceph
[22:41] <SamYaple> assuming you have an existing LV. i havent used LVM (by choice) in years now
[22:43] * penguinRaider_ (~KiKo@116.202.36.52) has joined #ceph
[22:43] <antongribok> yeah, I'm one of those few people that actually like LVM :)
[22:46] * gauravbafna (~gauravbaf@122.172.193.249) Quit (Ping timeout: 480 seconds)
[22:46] * Nicola-1_ (~Nicola-19@x4db30f9c.dyn.telefonica.de) Quit (Read error: Connection reset by peer)
[22:46] * Nicola-1980 (~Nicola-19@x4db30f9c.dyn.telefonica.de) has joined #ceph
[22:47] * penguinRaider__ (~KiKo@116.202.36.52) has joined #ceph
[22:50] <SamYaple> haha antongribok im just kidding. LVM is ok. it was _amazing_ years ago, but i feel like we need an LVM3 for it to be relevant again
[22:50] <SamYaple> and its just not going to happen i dont think
[22:51] <gwinger> Hi <SamYaple>
[22:51] <SamYaple> hey you
[22:51] <SamYaple> new questions?
[22:51] <gwinger> I am reading some confusing stuff quite this moment
[22:51] <gwinger> ;-)
[22:51] <gwinger> easy question ;-)
[22:51] * rendar (~I@host153-194-dynamic.18-79-r.retail.telecomitalia.it) has joined #ceph
[22:51] <SamYaple> ill be the judge of that
[22:52] <gwinger> wD WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
[22:52] <gwinger> -21 12.00000 root sata
[22:52] <gwinger> -22 4.00000 host de_pod0_sata
[22:52] <gwinger> 2 1.00000 osd.2 up 1.00000 1.00000
[22:52] <gwinger> 3 1.00000 osd.3 up 1.00000 1.00000
[22:52] <gwinger> 4 1.00000 osd.4 up 1.00000 1.00000
[22:52] <gwinger> 5 1.00000 osd.5 up 1.00000 1.00000
[22:52] <gwinger> -23 4.00000 host de_pod1_sata
[22:52] <gwinger> 8 1.00000 osd.8 up 1.00000 1.00000
[22:52] <gwinger> 9 1.00000 osd.9 up 1.00000 1.00000
[22:52] <gwinger> 10 1.00000 osd.10 up 1.00000 1.00000
[22:52] <SamYaple> make sure you use some pastebin
[22:52] <gwinger> 11 1.00000 osd.11 up 1.00000 1.00000
[22:52] <gwinger> -24 4.00000 host de_pod2_sata
[22:52] <gwinger> 14 1.00000 osd.14 up 1.00000 1.00000
[22:52] <SamYaple> dont paste to channel like this
[22:52] <gwinger> 15 1.00000 osd.15 up 1.00000 1.00000
[22:52] <gwinger> 16 1.00000 osd.16 up 1.00000 1.00000
[22:52] <gwinger> 17 1.00000 osd.17 up 1.00000 1.00000
[22:52] <gwinger> -1 6.00000 root ssd
[22:52] <gwinger> -2 2.00000 host de_pod0_ssd
[22:52] <gwinger> 0 1.00000 osd.0 up 1.00000 1.00000
[22:52] <gwinger> 1 1.00000 osd.1 up 1.00000 1.00000
[22:52] <gwinger> -3 2.00000 host de_pod1_ssd
[22:52] <gwinger> 6 1.00000 osd.6 up 1.00000 1.00000
[22:53] <gwinger> 7 1.00000 osd.7 up 1.00000 1.00000
[22:53] <gwinger> -4 2.00000 host de_pod2_ssd
[22:53] <gwinger> 12 1.00000 osd.12 up 1.00000 1.00000
[22:53] <gwinger> 13 1.00000 osd.13 up 1.00000 1.00000
[22:53] <gwinger> this is my old tree
[22:53] <gwinger> you see the data and the ssl pool
[22:53] <gwinger> everythingg is cell
[22:53] <gwinger> ok
[22:53] <gwinger> Now I found this in the docs
[22:53] <gwinger> Running instances from RBD has a couple of limitations:
[22:53] <gwinger> An additional NFS Primary Storage pool is required for running System VM?s
[22:53] <gwinger> Snapshotting RBD volumes is not possible (at this moment)
[22:53] <gwinger> Only one monitor can be configured
[22:53] <gwinger> Running instances from RBD has a couple of limitations:
[22:53] <gwinger> An additional NFS Primary Storage pool is required for running System VM?s
[22:53] <gwinger> Snapshotting RBD volumes is not possible (at this moment)
[22:53] <gwinger> Only one monitor can be configured
[22:53] <gwinger> Limitations?
[22:54] * penguinRaider_ (~KiKo@116.202.36.52) Quit (Ping timeout: 480 seconds)
[22:54] <SamYaple> gwinger: pastebin or links ot the docs. dont paste to channel
[22:54] <gwinger> Running instances from RBD has a couple of limitations:
[22:54] <gwinger> An additional NFS Primary Storage pool is required for running System VM?s
[22:54] <gwinger> Snapshotting RBD volumes is not possible (at this moment)
[22:54] <gwinger> Only one monitor can be configured
[22:54] <gwinger> Running instances from RBD has a couple of limitations:
[22:54] <gwinger> An additional NFS Primary Storage pool is required for running System VM?s
[22:54] * penguinRaider_ (~KiKo@116.202.36.52) has joined #ceph
[22:54] <gwinger> Snapshotting RBD volumes is not possible (at this moment)
[22:54] <gwinger> Only one monitor can be configured
[22:54] <via> thats a paddlin
[22:54] <gwinger> Running instances from RBD has a couple of limitations:
[22:54] <gwinger> An additional NFS Primary Storage pool is required for running System VM?s
[22:54] <gwinger> Snapshotting RBD volumes is not possible (at this moment)
[22:54] <gwinger> Only one monitor can be configured
[22:54] <gwinger> Running Instances from RBD has a couple of limitations
[22:54] <gwinger> Sorry
[22:54] <gwinger> http://dachary.org/loic/ceph-doc/rbd/rbd-cloudstack/
[22:55] <SamYaple> wow gwinger
[22:55] <SamYaple> thats amazing
[22:55] <gwinger> The write block "Limitations"
[22:55] <SamYaple> thats _really_ old docs
[22:55] <gwinger> ok?
[22:55] <gwinger> Yeah, what I need to know
[22:56] * bniver (~bniver@nat-pool-bos-u.redhat.com) Quit (Remote host closed the connection)
[22:56] <gwinger> should I add a user for the ssl pool now and deploy it completely to cloudstack as primary storage?
[22:56] <gwinger> Or do I need to respect the "old docs"?
[22:56] <gwinger> for the SSD pool
[22:56] <gwinger> nit ssl
[22:57] <gwinger> typo ..
[22:57] <SamYaple> that sounds like a cloudstack specific question. i dont have an answer for you there. look at more recent docs is my suggestion
[22:57] <SamYaple> those docs you link are ~3 years old
[22:57] <gwinger> ok
[22:58] * EinstCrazy (~EinstCraz@58.247.119.250) has joined #ceph
[22:58] <gwinger> do you think the limitation is obsolete now?
[22:58] * penguinRaider__ (~KiKo@116.202.36.52) Quit (Ping timeout: 480 seconds)
[22:59] <gwinger> the one Snapshotting RBD volumes is not possible (at this moment)
[22:59] <SamYaple> rbd snapshotting is possible, i dont know if its possible through cloudstack though
[23:06] * EinstCrazy (~EinstCraz@58.247.119.250) Quit (Ping timeout: 480 seconds)
[23:16] * scg (~zscg@valis.gnu.org) Quit (Ping timeout: 480 seconds)
[23:18] * rdias (~rdias@2001:8a0:749a:d01:e05f:7d9:e336:c31c) Quit (Ping timeout: 480 seconds)
[23:21] * Maariu5_ (~Chrissi_@tor1e1.privacyfoundation.ch) has joined #ceph
[23:23] * penguinRaider__ (~KiKo@116.202.36.52) has joined #ceph
[23:24] * rdias (~rdias@bl7-92-98.dsl.telepac.pt) has joined #ceph
[23:30] * penguinRaider_ (~KiKo@116.202.36.52) Quit (Ping timeout: 480 seconds)
[23:34] * penguinRaider_ (~KiKo@116.202.36.52) has joined #ceph
[23:36] * nils_ (~nils_@doomstreet.collins.kg) Quit (Quit: This computer has gone to sleep)
[23:42] * penguinRaider__ (~KiKo@116.202.36.52) Quit (Ping timeout: 480 seconds)
[23:50] * penguinRaider__ (~KiKo@116.202.36.52) has joined #ceph
[23:50] * visbits (~textual@8.29.138.28) has joined #ceph
[23:50] * Maariu5_ (~Chrissi_@06SAAD18P.tor-irc.dnsbl.oftc.net) Quit ()
[23:51] * Frostshifter (~Quackie@176.10.99.208) has joined #ceph
[23:53] * rendar (~I@host153-194-dynamic.18-79-r.retail.telecomitalia.it) Quit (Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!)
[23:58] * penguinRaider_ (~KiKo@116.202.36.52) Quit (Ping timeout: 480 seconds)

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.