#ceph IRC Log

Index

IRC Log for 2014-03-10

Timestamps are in GMT/BST.

[0:00] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[0:02] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) has joined #ceph
[0:05] * Boltsky (~textual@cpe-198-72-138-106.socal.res.rr.com) Quit (Quit: Textual IRC Client: www.textualapp.com)
[0:07] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has joined #ceph
[0:07] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has joined #ceph
[0:07] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has left #ceph
[0:10] * sarob (~sarob@2601:9:7080:13a:c82c:56c0:3ea:2a07) has joined #ceph
[0:19] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[0:20] * yanzheng (~zhyan@134.134.137.71) has joined #ceph
[0:26] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) Quit (Ping timeout: 480 seconds)
[0:35] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[0:36] * lightspeed (~lightspee@2001:8b0:16e:1:216:eaff:fe59:4a3c) has joined #ceph
[0:38] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has left #ceph
[0:49] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[0:51] * mkoderer (uid11949@id-11949.ealing.irccloud.com) Quit (Quit: Connection closed for inactivity)
[0:54] * lx0 (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[1:07] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[1:09] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[1:25] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has joined #ceph
[1:25] * zidarsk8 (~zidar@89-212-126-248.dynamic.t-2.net) has left #ceph
[1:27] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) Quit (Quit: Ja odoh a vi sta 'ocete...)
[1:27] * yanzheng (~zhyan@134.134.137.71) Quit (Remote host closed the connection)
[1:30] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[1:33] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[1:36] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit ()
[1:36] * LeaChim (~LeaChim@host86-166-182-74.range86-166.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[1:47] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[1:48] * LeaChim (~LeaChim@host86-166-182-74.range86-166.btcentralplus.com) has joined #ceph
[1:56] * sarob (~sarob@2601:9:7080:13a:c82c:56c0:3ea:2a07) Quit (Remote host closed the connection)
[1:57] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[2:04] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[2:04] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[2:12] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) has joined #ceph
[2:16] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[2:17] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[2:18] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[2:18] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) has joined #ceph
[2:22] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[2:23] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[2:28] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[2:36] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[2:44] * LeaChim (~LeaChim@host86-166-182-74.range86-166.btcentralplus.com) Quit (Ping timeout: 480 seconds)
[2:48] * dlan (~dennis@116.228.88.131) has joined #ceph
[2:50] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[2:53] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[2:54] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[2:59] * sarob (~sarob@2601:9:7080:13a:d1b1:7efd:918c:233) has joined #ceph
[3:00] * dlan (~dennis@116.228.88.131) has left #ceph
[3:02] * Boltsky (~textual@cpe-198-72-138-106.socal.res.rr.com) has joined #ceph
[3:05] * haomaiwa_ (~haomaiwan@49.4.189.43) Quit (Remote host closed the connection)
[3:05] * erkules (~erkules@port-92-193-5-143.dynamic.qsc.de) has joined #ceph
[3:06] * haomaiwang (~haomaiwan@219-87-173-15.static.tfn.net.tw) has joined #ceph
[3:11] * sarob (~sarob@2601:9:7080:13a:d1b1:7efd:918c:233) Quit (Ping timeout: 480 seconds)
[3:12] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[3:12] * carif (~mcarifio@htr06-1-82-227-229-89.fbx.proxad.net) has joined #ceph
[3:12] * erkules_ (~erkules@port-92-193-97-86.dynamic.qsc.de) Quit (Ping timeout: 480 seconds)
[3:16] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[3:20] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[3:23] * BillK (~BillK-OFT@58-7-107-254.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[3:24] * BillK (~BillK-OFT@106-68-241-113.dyn.iinet.net.au) has joined #ceph
[3:24] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[3:29] * ponyofdeath (~vladi@cpe-76-167-201-214.san.res.rr.com) Quit (Quit: Lost terminal)
[3:29] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[3:33] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[3:34] * Cube (~Cube@66-87-67-83.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[3:36] * shang (~ShangWu@175.41.48.77) has joined #ceph
[3:36] * ponyofdeath (~vladi@cpe-76-167-201-214.san.res.rr.com) has joined #ceph
[3:37] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[3:41] * haomaiwa_ (~haomaiwan@117.79.232.197) has joined #ceph
[3:48] * haomaiwang (~haomaiwan@219-87-173-15.static.tfn.net.tw) Quit (Ping timeout: 480 seconds)
[3:53] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Quit: Leaving.)
[3:54] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) has joined #ceph
[4:09] * kaizh (~kaizh@c-50-131-203-4.hsd1.ca.comcast.net) Quit ()
[4:13] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[4:22] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[4:24] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[4:26] * jrcresawn (~jrcresawn@ip24-251-38-21.ph.ph.cox.net) has joined #ceph
[4:29] <jrcresawn> I have a network question. I have eight 1-Gb/s NICs in a Dell storage server with two SSDs and twelve 2-TB disks. I'm debating if I should configure LACP for these NICs or I wondered if there was a better way to configure the NICs. For instance, could I assign each osd its own NIC?
[4:30] * sarob (~sarob@2601:9:7080:13a:3dc1:1901:397b:d630) has joined #ceph
[4:30] <jrcresawn> Correction: The server in question has ten 1-Gb/s NICs.
[4:35] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) has joined #ceph
[4:35] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) Quit (Ping timeout: 480 seconds)
[4:39] <cowbar> jrcresawn: why not just get a single 10g port for each machine?
[4:39] * ircolle (~Adium@2601:1:8380:2d9:c4f7:ab3d:4ae2:204a) has joined #ceph
[4:41] <cowbar> though I don't know cost per port for 1g vs 10g linespeed-capable switches, so perhaps thats the reasoning
[4:42] <cowbar> LACP is likely the way to go though. into two separate bundles, one for client-facing and one for cluster-facing, so that background cluster tasks don't impact your client performance.
[4:42] <jrcresawn> That would be simpler. The main reason is money. I'm trying to work with the hardware we have today. One of those pieces of hardware is a 48-port gigabit switch.
[4:42] * yuriw (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) has joined #ceph
[4:43] <cowbar> and connect each bundle to a different network/vlan
[4:44] <jrcresawn> My plan is to combine 8 NICs together with LACP and using tagged VLANs to direct traffic to the correct network.
[4:45] <cowbar> still having a split between cluster and client networks, just logically?
[4:45] <jrcresawn> Yes, one VLAN for cluster and another for client.
[4:46] * sarob (~sarob@2601:9:7080:13a:3dc1:1901:397b:d630) Quit (Ping timeout: 480 seconds)
[4:46] <cowbar> I'm no expert on such things, but it seems to me like having two groups of 4 NICs each would be better, as it makes it so the client network can not saturate the cluster network, and vice-versa..
[4:47] * yuriw2 (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) has joined #ceph
[4:47] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) has joined #ceph
[4:48] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) Quit ()
[4:48] * yuriw1 (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[4:50] <jrcresawn> I too am not an expert on network design.
[4:50] * yuriw (~Adium@c-71-202-126-141.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[4:51] <cowbar> set it up one way and do some testing with trying to saturate things, then set it up the other way and do the same? :)
[4:51] <jrcresawn> Agreed. That's the only way to be sure.
[4:52] <cowbar> how big is the cluster you're setting up?
[4:54] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) has joined #ceph
[4:55] <jrcresawn> Currently I have one server, but I hope to receive approval to purchase two more. Actually, that brings up another question. Is there any problem having different disk sizes within the cluster?
[4:58] <cowbar> I believe so but i'm not certain. new to ceph myself.
[5:03] <jrcresawn> OK, so you believe there is a problem with different size disks?
[5:04] <cowbar> fairly certain you can have multiple sizes, but you need to be aware of the IO requirements and how they can scale with the amount of data, else you can run into bottlenecks for the larger drives
[5:04] <cowbar> and I don't know the details of how you want to set things up to avoid such issues.
[5:05] <cowbar> I recall a page in the docs awhile back that mentioned such things. let me see if I can turn that up for you.
[5:09] * shang (~ShangWu@175.41.48.77) Quit (Read error: Operation timed out)
[5:11] <cowbar> jrcresawn: this is where I read it, see the section about weights: http://ceph.com/docs/master/rados/operations/crush-map/#crush-map-bucket-hierarchy
[5:11] <cowbar> but I myself have not setup a cluster with different-size disks. my only small test cluster used identical disks in all osds
[5:11] <cowbar> so no direct experience with it.
[5:18] * Vacum (~vovo@i59F792CE.versanet.de) has joined #ceph
[5:21] <jrcresawn> cowbar: thanks for the URL.
[5:25] * Vacum_ (~vovo@88.130.200.59) Quit (Ping timeout: 480 seconds)
[5:36] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[5:37] * carif (~mcarifio@htr06-1-82-227-229-89.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[5:38] * ircolle (~Adium@2601:1:8380:2d9:c4f7:ab3d:4ae2:204a) Quit (Quit: Leaving.)
[5:38] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[5:38] * The_Bishop (~bishop@2001:470:50b6:0:31e0:67e4:5a31:49f4) Quit (Quit: Wer zum Teufel ist dieser Peer? Wenn ich den erwische dann werde ich ihm mal die Verbindung resetten!)
[5:39] * ircolle (~Adium@c-67-172-132-222.hsd1.co.comcast.net) has joined #ceph
[5:40] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[5:42] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Read error: Connection reset by peer)
[5:56] * lxo (~aoliva@lxo.user.oftc.net) Quit (Ping timeout: 480 seconds)
[5:56] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) has joined #ceph
[5:58] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) has joined #ceph
[6:01] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[6:04] * valeech (~valeech@pool-71-171-123-210.clppva.fios.verizon.net) Quit (Quit: valeech)
[6:04] * dpippenger (~riven@cpe-198-72-157-189.socal.res.rr.com) has joined #ceph
[6:08] * The_Bishop (~bishop@2001:470:50b6:0:31e0:67e4:5a31:49f4) has joined #ceph
[6:15] * jrcresawn (~jrcresawn@ip24-251-38-21.ph.ph.cox.net) has left #ceph
[6:16] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) Quit (Remote host closed the connection)
[6:17] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) has joined #ceph
[6:17] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[6:19] * ircolle (~Adium@c-67-172-132-222.hsd1.co.comcast.net) Quit (Quit: Leaving.)
[6:21] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[6:27] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) has joined #ceph
[6:45] * geraintjones (~geraint@222-152-77-45.jetstream.xtra.co.nz) Quit (Quit: geraintjones)
[6:53] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) Quit (Quit: Leaving.)
[6:57] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[6:57] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[6:59] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) Quit (Ping timeout: 480 seconds)
[7:05] * lavi (~lavi@eth-seco21th2-46-193-64-32.wb.wifirst.net) Quit (Remote host closed the connection)
[7:09] * mdxi (~mdxi@50-199-109-154-static.hfc.comcastbusiness.net) has joined #ceph
[7:13] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) Quit (Quit: Leaving.)
[7:15] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) has joined #ceph
[7:24] * carif (~mcarifio@htr06-1-82-227-229-89.fbx.proxad.net) has joined #ceph
[7:24] * shang (~ShangWu@175.41.48.77) has joined #ceph
[7:30] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[7:36] * tiger (~textual@58.213.102.114) has joined #ceph
[7:36] * yguang11 (~yguang11@vpn-nat.peking.corp.yahoo.com) Quit (Read error: Connection reset by peer)
[7:36] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) has joined #ceph
[7:38] <tiger> anyone using hadoop with cephfs?
[7:38] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) Quit (Quit: Leaving.)
[7:40] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) has joined #ceph
[7:46] * AfC (~andrew@2407:7800:400:1011:2ad2:44ff:fe08:a4c) Quit (Quit: Leaving.)
[7:53] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has left #ceph
[7:59] * carif (~mcarifio@htr06-1-82-227-229-89.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[8:07] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[8:09] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) has joined #ceph
[8:12] * ajazdzewski (~quassel@lpz-66.sprd.net) has joined #ceph
[8:16] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) Quit (Read error: No route to host)
[8:25] * imriz (~imriz@82.81.163.130) has joined #ceph
[8:30] * sarob (~sarob@2601:9:7080:13a:487:bb6f:5059:13) has joined #ceph
[8:36] * shang (~ShangWu@175.41.48.77) Quit (Ping timeout: 480 seconds)
[8:36] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[8:46] * sarob (~sarob@2601:9:7080:13a:487:bb6f:5059:13) Quit (Ping timeout: 480 seconds)
[8:47] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) Quit (Quit: Leaving)
[8:50] * gaveen (~gaveen@220.247.234.28) has joined #ceph
[8:52] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[8:55] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) Quit (Remote host closed the connection)
[8:56] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) has joined #ceph
[8:58] * thb (~me@0001bd58.user.oftc.net) has joined #ceph
[9:00] * Sysadmin88 (~IceChat77@176.254.32.31) Quit (Quit: Hard work pays off in the future, laziness pays off now)
[9:00] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[9:00] * ChanServ sets mode +v andreask
[9:03] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) has joined #ceph
[9:04] * shang (~ShangWu@175.41.48.77) has joined #ceph
[9:04] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) Quit (Read error: Connection reset by peer)
[9:06] <jerker> For your information, running Esc4 on top of ZVOL/ZFS, is both slow and unstable. Native ZFS I do not know yet.
[9:06] <jerker> (With Ceph on top of Ext4)
[9:15] <sputnik13> ??
[9:15] <sputnik13> ceph on ext4 on zfs ?
[9:15] * sputnik13 is confused
[9:17] * shang (~ShangWu@175.41.48.77) Quit (Ping timeout: 480 seconds)
[9:17] * rendar (~s@95.238.179.204) has joined #ceph
[9:17] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) Quit (Read error: No route to host)
[9:18] * yguang11 (~yguang11@2406:2000:ef96:e:a453:57de:7900:7069) has joined #ceph
[9:24] * thomnico (~thomnico@2a01:e35:8b41:120:c533:3678:1051:e7d2) has joined #ceph
[9:25] <jerker> sputnik13: I was experimenting with running Ceph on my existing ZFS boxes. I could not run it natively on ZFS (need to recompile Ceph or something) but it started up fine and ran during the weekend on a ZVOL (blockdevice, like a loopback device) with Ext4 on top. And then Ceph-OSD running on the Ext4.
[9:27] <sputnik13> jerker: don't mean to be judgemental, but that sounds like a bad idea
[9:30] * sarob (~sarob@2601:9:7080:13a:25e6:7446:e091:6c8b) has joined #ceph
[9:30] * shang (~ShangWu@114-32-21-24.HINET-IP.hinet.net) has joined #ceph
[9:33] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) has joined #ceph
[9:38] * sarob (~sarob@2601:9:7080:13a:25e6:7446:e091:6c8b) Quit (Ping timeout: 480 seconds)
[9:48] * shang_ (~ShangWu@175.41.48.77) has joined #ceph
[9:49] * geraintjones (~geraint@222-152-77-45.jetstream.xtra.co.nz) has joined #ceph
[9:50] * allsystemsarego (~allsystem@188.26.167.156) has joined #ceph
[9:51] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) Quit (Quit: Leaving)
[9:52] * Siva_ (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[9:53] * shang (~ShangWu@114-32-21-24.HINET-IP.hinet.net) Quit (Ping timeout: 480 seconds)
[9:53] * Siva (~sivat@generalnat.eglbp.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[9:53] * Siva_ is now known as Siva
[9:53] * The_Bishop (~bishop@2001:470:50b6:0:31e0:67e4:5a31:49f4) Quit (Ping timeout: 480 seconds)
[9:55] * tiger (~textual@58.213.102.114) Quit (Quit: Textual IRC Client: www.textualapp.com)
[10:00] <jerker> sputnik13: Ceph on native ZFS is interesting because of checksums and compression.
[10:00] <jerker> sputnik13: And integrated SSD cache and log. But XFS with bcache is most probably faster.
[10:02] * The_Bishop (~bishop@2001:470:50b6:0:ac66:1a90:db0d:ffec) has joined #ceph
[10:04] <sputnik13> I'm a big fan of ZFS in general, but it seems to me that adds more complexity, probably more than is worthwhile
[10:04] <jerker> Indeed.
[10:04] <cowbar> exactly my thoughts on zfs+ceph.
[10:04] <sputnik13> ZFS expects control over multiple spindles to scale out throughput and IO, ceph expects the same things
[10:05] * geraintjones (~geraint@222-152-77-45.jetstream.xtra.co.nz) Quit (Quit: geraintjones)
[10:05] <jerker> sputnik13: In the long run I hope for stable Btrfs with compression. Maybe it is there already? I have not tried.
[10:05] <sputnik13> what benefit you'd get out of doing that at two levels I don't know
[10:05] <jerker> sputnik13: This time just experimentation.
[10:06] <sputnik13> I'm shying away from btrfs... I shy away from anything that isn't "production ready"... too many other things to do :(
[10:06] <jerker> sputnik13: If one wishes to run Ceph on native ZFS then one pool for each drive like for XFS/Ext4.
[10:07] <sputnik13> erasure coding is the most interesting thing right now, but I'm on the fence as to whether I want to deploy the first released code with erasure coding
[10:07] <jerker> sputnik13: Totally agree.
[10:07] <sputnik13> but if you do that what benefit do you derive from ZFS
[10:07] <sputnik13> other than snapshot
[10:07] <jerker> sputnik13: compression checksums
[10:07] <sputnik13> compression on 64kb blocks?
[10:08] <sputnik13> doesn't ceph store 64kb blocks? better to compress the file before it gets broken down to blocks/objects I would think
[10:08] <jerker> sputnik13: No idea actually. For some of my data I get 3.5x compression on native ZFS when doing rsync into it and I was hoping for at least some of that...
[10:09] <sputnik13> when you're storing a 100MB file natively on ZFS, it can try to compress that down and if it's text data it'll compress well
[10:09] <sputnik13> if you store that in 64KB chunks, then you're compressing 64KB individual chunks, how much that will compress down is I think less predictable and questionable
[10:10] <jerker> sputnik13: yep. One other problem is that ZFS on Linux is not really ready for prime time yet. I have been running it since it went mountable but well, it do not crash as often, but it still does. :/
[10:11] <sputnik13> the idea with compression (lossless) is to find an algorithm or pattern that allows you to represent larger data sets with a smaller amount of data using said algorithm/pattern, but if you're taking 64KB chunks, how much can you really do, I wonder
[10:11] <jerker> sputnik13: I have ~50 million files in one of the file systems and I guess that is not very common. It is rock solid on Ext4 though.
[10:11] <sputnik13> I've only used ZFS on solaris/openindiana
[10:12] <sputnik13> solaris ftw :-)
[10:13] <jerker> sputnik13: Solaris is/was very solid, but I have always been into the migrate from solaris to linux business, working around the flaws in Linux when they pop up. ZFS is indeed better on Solaris and also better on FreeBSD than on Linux, still.
[10:17] <sputnik13> imho solaris is way better for the enterprise space than linux in so many ways, it's just damn expensive
[10:17] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) has joined #ceph
[10:17] <sputnik13> anyway, back to work for me, gl with your ceph adventures
[10:21] * humbolt (~elias@213-33-6-247.adsl.highway.telekom.at) Quit (Read error: Connection reset by peer)
[10:23] <jerker> gl&hf
[10:25] * TMM (~hp@sams-office-nat.tomtomgroup.com) has joined #ceph
[10:27] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) has joined #ceph
[10:29] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[10:30] * sarob (~sarob@2601:9:7080:13a:cc4:87e9:624a:69ee) has joined #ceph
[10:35] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) Quit (Ping timeout: 480 seconds)
[10:36] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[10:39] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[10:49] * AfC (~andrew@2001:44b8:31cb:d400:6e88:14ff:fe33:2a9c) Quit (Quit: Leaving.)
[11:02] * mattt (~textual@94.236.7.190) has joined #ceph
[11:07] * sarob (~sarob@2601:9:7080:13a:cc4:87e9:624a:69ee) Quit (Ping timeout: 480 seconds)
[11:08] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) has joined #ceph
[11:11] * hjjg (~hg@p3EE307D0.dip0.t-ipconnect.de) has joined #ceph
[11:13] * thomnico (~thomnico@2a01:e35:8b41:120:c533:3678:1051:e7d2) Quit (Quit: Ex-Chat)
[11:13] * thomnico (~thomnico@2a01:e35:8b41:120:c533:3678:1051:e7d2) has joined #ceph
[11:17] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[11:17] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[11:30] * sarob (~sarob@2601:9:7080:13a:c01a:bc7c:7c8e:3857) has joined #ceph
[11:36] * shang_ (~ShangWu@175.41.48.77) Quit (Quit: Ex-Chat)
[12:02] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[12:05] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[12:06] * sarob (~sarob@2601:9:7080:13a:c01a:bc7c:7c8e:3857) Quit (Ping timeout: 480 seconds)
[12:11] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) Quit (Quit: My Mac Pro has gone to sleep. ZZZzzz???)
[12:12] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[12:18] * jonasschneider (~jonasschn@HSI-KBW-046-005-233-246.hsi8.kabel-badenwuerttemberg.de) has joined #ceph
[12:21] * funnel (~funnel@0001c7d4.user.oftc.net) has joined #ceph
[12:30] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[12:35] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[12:36] * `jpg (~jpg@ppp121-44-218-235.lns20.syd7.internode.on.net) has joined #ceph
[12:47] * fdmanana (~fdmanana@bl9-168-202.dsl.telepac.pt) has joined #ceph
[12:48] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) has joined #ceph
[12:51] * jonasschneider (~jonasschn@HSI-KBW-046-005-233-246.hsi8.kabel-badenwuerttemberg.de) Quit (Remote host closed the connection)
[12:55] * LeaChim (~LeaChim@host86-166-182-74.range86-166.btcentralplus.com) has joined #ceph
[12:56] * Cataglottism (~Cataglott@dsl-087-195-030-184.solcon.nl) has joined #ceph
[13:04] * diegows (~diegows@190.190.5.238) has joined #ceph
[13:07] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[13:09] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[13:11] * ircuser-1 (~ircuser-1@35.222-62-69.ftth.swbr.surewest.net) Quit (Ping timeout: 480 seconds)
[13:11] * c74d (~c74d@2002:4404:712c:0:4932:35fb:550:916) Quit (Remote host closed the connection)
[13:11] * c74d (~c74d@2002:4404:712c:0:bd2c:2122:b6f4:9077) has joined #ceph
[13:11] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[13:13] * ChrisNBlum (~Adium@dhcp-ip-230.dorf.rwth-aachen.de) has joined #ceph
[13:15] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[13:17] * lianghaoshen (~slhhust@113.240.124.219) has joined #ceph
[13:18] * lianghaoshen (~slhhust@113.240.124.219) Quit ()
[13:25] * KevinPerks (~Adium@cpe-066-026-252-218.triad.res.rr.com) has joined #ceph
[13:26] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) has joined #ceph
[13:27] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Quit: Siva)
[13:30] * gaveen (~gaveen@220.247.234.28) Quit (Quit: Leaving)
[13:30] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[13:37] * funnel is now known as Guest2783
[13:38] * Guest2783 is now known as funnel
[13:41] * hybrid512 (~walid@195.200.167.70) Quit (Quit: Leaving.)
[13:43] * hybrid512 (~walid@195.200.167.70) has joined #ceph
[13:44] * hybrid512 (~walid@195.200.167.70) Quit ()
[13:44] * hybrid512 (~walid@195.200.167.70) has joined #ceph
[13:45] * wschulze (~wschulze@cpe-72-229-37-201.nyc.res.rr.com) has joined #ceph
[13:49] * codice (~toodles@97-94-175-73.static.mtpk.ca.charter.com) Quit (Remote host closed the connection)
[13:49] * codice (~toodles@97-94-175-73.static.mtpk.ca.charter.com) has joined #ceph
[13:53] * ircuser-1 (~ircuser-1@35.222-62-69.ftth.swbr.surewest.net) has joined #ceph
[14:01] * BillK (~BillK-OFT@106-68-241-113.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[14:02] * BillK (~BillK-OFT@124-148-93-148.dyn.iinet.net.au) has joined #ceph
[14:04] * hybrid512 (~walid@195.200.167.70) Quit (Quit: Leaving.)
[14:05] * c74d (~c74d@2002:4404:712c:0:bd2c:2122:b6f4:9077) Quit (Remote host closed the connection)
[14:05] * Cataglottism (~Cataglott@dsl-087-195-030-184.solcon.nl) Quit (Ping timeout: 480 seconds)
[14:05] * c74d (~c74d@2002:4404:712c:0:7034:6ce:d8be:15a8) has joined #ceph
[14:06] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Read error: Operation timed out)
[14:08] * hybrid512 (~walid@195.200.167.70) has joined #ceph
[14:09] * hybrid512 (~walid@195.200.167.70) Quit ()
[14:09] * hybrid512 (~walid@195.200.167.70) has joined #ceph
[14:12] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has left #ceph
[14:13] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[14:18] * hasues (~hazuez@108-236-232-243.lightspeed.knvltn.sbcglobal.net) has joined #ceph
[14:21] * clayb (~kvirc@199.172.169.97) has joined #ceph
[14:23] * japuzzo (~japuzzo@ool-4570886e.dyn.optonline.net) has joined #ceph
[14:25] * Siva (~sivat@117.192.33.208) has joined #ceph
[14:30] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[14:33] * Siva (~sivat@117.192.33.208) Quit (Ping timeout: 480 seconds)
[14:35] * adam__ (~adam@rincewind.universalconflicts.com) Quit (Quit: leaving)
[14:35] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) has joined #ceph
[14:35] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) has joined #ceph
[14:35] * jedanbik (~jedanbik@dhcp152-54-6-141.wireless.europa.renci.org) has joined #ceph
[14:36] * sroy (~sroy@207.96.182.162) has joined #ceph
[14:41] * lofejndif (~lsqavnbok@hessel0.torservers.net) has joined #ceph
[14:41] * valeech (~valeech@ip72-205-7-86.dc.dc.cox.net) has joined #ceph
[14:43] * zoltan (~zoltan@2001:620:20:222:a40a:c74d:f691:b0ca) has joined #ceph
[14:43] <zoltan> hi
[14:44] <zoltan> playing around with 0.77, especially with tiering currently, and it's ok that I can specify a pool to be a tier for an other pool
[14:44] <zoltan> but what's the easiest way to limit a pool to certain osds?
[14:52] * hjjg_ (~hg@p3EE3262E.dip0.t-ipconnect.de) has joined #ceph
[14:53] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) Quit (Quit: ZNC - http://znc.in)
[14:54] * janos (~messy@static-71-176-211-4.rcmdva.fios.verizon.net) has joined #ceph
[14:54] * hjjg (~hg@p3EE307D0.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[14:56] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) has joined #ceph
[14:57] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) has joined #ceph
[15:00] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) Quit ()
[15:03] <zoltan> ah, ok, found out
[15:07] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[15:11] * Psi-Jack_ (~Psi-Jack@psi-jack.user.oftc.net) has joined #ceph
[15:12] * BillK (~BillK-OFT@124-148-93-148.dyn.iinet.net.au) Quit (Ping timeout: 480 seconds)
[15:14] * lofejndif (~lsqavnbok@9IOAACFGF.tor-irc.dnsbl.oftc.net) Quit (Quit: gone)
[15:16] * alfredodeza (~alfredode@198.206.133.89) has left #ceph
[15:18] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) has joined #ceph
[15:21] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) has joined #ceph
[15:26] * odyssey4me (~odyssey4m@165.233.71.2) has joined #ceph
[15:30] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[15:31] * dmsimard (~Adium@108.163.152.2) has joined #ceph
[15:38] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[15:40] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) Quit (Quit: Leaving.)
[15:40] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[15:40] * ChanServ sets mode +v andreask
[15:41] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[15:43] * gregmark (~Adium@68.87.42.115) has joined #ceph
[15:55] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[15:55] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) has joined #ceph
[15:55] * sputnik1_ (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit ()
[15:56] * ChrisNBlum (~Adium@dhcp-ip-230.dorf.rwth-aachen.de) Quit (Read error: Connection reset by peer)
[15:57] * ChrisNBlum (~Adium@dhcp-ip-230.dorf.rwth-aachen.de) has joined #ceph
[15:59] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) has joined #ceph
[16:01] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) Quit (Ping timeout: 480 seconds)
[16:03] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) Quit (Quit: Leaving.)
[16:05] * t0rn (~ssullivan@2607:fad0:32:a02:d227:88ff:fe02:9896) has joined #ceph
[16:17] * Kioob`Taff (~plug-oliv@local.plusdinfo.com) has joined #ceph
[16:21] * ChrisNBlum (~Adium@dhcp-ip-230.dorf.rwth-aachen.de) Quit (Quit: Leaving.)
[16:30] * sarob (~sarob@2601:9:7080:13a:a414:9651:5f7d:4763) has joined #ceph
[16:34] <odyssey4me> does anyone know how to make an rbd block device get rescanned after a resize?
[16:34] <odyssey4me> I've increased its size, but the new size won't show
[16:35] <fghaas> odyssey4me: kernel rbd? unmap and then map
[16:36] <odyssey4me> odd, tried that - didn't work
[16:36] <fghaas> what didn't work? unmapping?
[16:36] * rmoe (~quassel@173-228-89-134.dsl.static.sonic.net) Quit (Read error: Operation timed out)
[16:36] <odyssey4me> fgaas - nope, unmapped and remapped... parted still only sees the original size
[16:38] <odyssey4me> ah, interesting - unmapped and the device mapper still sees the device
[16:38] * sarob (~sarob@2601:9:7080:13a:a414:9651:5f7d:4763) Quit (Ping timeout: 480 seconds)
[16:41] * shang (~ShangWu@220-135-203-169.HINET-IP.hinet.net) has joined #ceph
[16:43] * Cube (~Cube@66-87-64-6.pools.spcsdns.net) has joined #ceph
[16:44] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[16:48] * sputnik1_ (~sputnik13@207.8.121.241) has joined #ceph
[16:49] * nwat (~textual@c-50-131-197-174.hsd1.ca.comcast.net) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[16:54] * zerick (~eocrospom@190.114.248.76) has joined #ceph
[16:56] * ircolle (~Adium@2601:1:8380:2d9:787d:3113:d2c6:7618) has joined #ceph
[16:57] * gregsfortytwo (~Adium@2607:f298:a:607:3d79:585d:5557:ffbc) Quit (Quit: Leaving.)
[16:57] * rmoe (~quassel@12.164.168.117) has joined #ceph
[16:58] * gregsfortytwo (~Adium@2607:f298:a:607:f85f:560e:6e7c:86b5) has joined #ceph
[17:02] * rmoe_ (~quassel@12.164.168.117) has joined #ceph
[17:03] * angdraug (~angdraug@12.164.168.117) has joined #ceph
[17:05] * alram (~alram@38.122.20.226) has joined #ceph
[17:05] * zackc (~zackc@0001ba60.user.oftc.net) Quit (Ping timeout: 480 seconds)
[17:06] * rmoe (~quassel@12.164.168.117) Quit (Ping timeout: 480 seconds)
[17:07] * thomnico (~thomnico@2a01:e35:8b41:120:c533:3678:1051:e7d2) Quit (Quit: Ex-Chat)
[17:09] * ajazdzewski (~quassel@lpz-66.sprd.net) Quit (Remote host closed the connection)
[17:09] * xarses (~andreww@c-24-23-183-44.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[17:15] * ChrisNBlum (~Adium@dhcp-ip-230.dorf.rwth-aachen.de) has joined #ceph
[17:16] * imriz (~imriz@82.81.163.130) Quit (Ping timeout: 480 seconds)
[17:17] * thomnico (~thomnico@2a01:e35:8b41:120:c533:3678:1051:e7d2) has joined #ceph
[17:17] * thomnico (~thomnico@2a01:e35:8b41:120:c533:3678:1051:e7d2) Quit ()
[17:21] * jeff-YF (~jeffyf@67.23.117.122) has joined #ceph
[17:21] * yanzheng (~zhyan@jfdmzpr01-ext.jf.intel.com) Quit (Remote host closed the connection)
[17:27] * BManojlovic (~steki@91.195.39.5) Quit (Quit: Ja odoh a vi sta 'ocete...)
[17:30] * BManojlovic (~steki@91.195.39.5) has joined #ceph
[17:30] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[17:31] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) has joined #ceph
[17:33] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[17:34] * bandrus (~Adium@75.5.250.121) has joined #ceph
[17:35] * nwat (~textual@eduroam-246-150.ucsc.edu) has joined #ceph
[17:36] * xmltok (~xmltok@cpe-76-90-130-148.socal.res.rr.com) Quit (Quit: Leaving...)
[17:38] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Ping timeout: 480 seconds)
[17:39] * bandrus1 (~Adium@75.5.250.121) has joined #ceph
[17:39] * bandrus (~Adium@75.5.250.121) Quit (Read error: No route to host)
[17:40] * bandrus1 (~Adium@75.5.250.121) Quit ()
[17:40] * bandrus (~Adium@adsl-75-5-250-121.dsl.scrm01.sbcglobal.net) has joined #ceph
[17:44] * BManojlovic (~steki@91.195.39.5) Quit (Ping timeout: 480 seconds)
[17:44] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) has joined #ceph
[17:50] * ajazdzewski (~quassel@lpz-66.sprd.net) has joined #ceph
[17:54] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) Quit (Remote host closed the connection)
[17:55] * effka (~Adium@193.174.25.30) has joined #ceph
[17:55] * xarses (~andreww@12.164.168.117) has joined #ceph
[17:59] * effka (~Adium@193.174.25.30) has left #ceph
[17:59] * effka (~Adium@193.174.25.30) has joined #ceph
[17:59] * effka (~Adium@193.174.25.30) has left #ceph
[17:59] * zoltan (~zoltan@2001:620:20:222:a40a:c74d:f691:b0ca) Quit (Ping timeout: 480 seconds)
[18:01] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) has joined #ceph
[18:02] * effka1984 (~Adium@193.174.25.30) has joined #ceph
[18:02] * effka1984 (~Adium@193.174.25.30) Quit ()
[18:03] * mattt (~textual@94.236.7.190) Quit (Ping timeout: 480 seconds)
[18:04] * c74d (~c74d@2002:4404:712c:0:7034:6ce:d8be:15a8) Quit (Remote host closed the connection)
[18:04] * c74d (~c74d@2002:4404:712c:0:281c:caf6:40c0:e857) has joined #ceph
[18:06] * ajazdzewski (~quassel@lpz-66.sprd.net) Quit (Read error: Operation timed out)
[18:07] * ajazdzewski (~quassel@lpz-66.sprd.net) has joined #ceph
[18:08] * zoltan (~zoltan@2001:620:20:16:f49b:d96f:edc1:a2c4) has joined #ceph
[18:09] * xarses (~andreww@12.164.168.117) Quit (Read error: No route to host)
[18:09] * xarses (~andreww@12.164.168.117) has joined #ceph
[18:10] * ajazdzewski (~quassel@lpz-66.sprd.net) Quit (Read error: Operation timed out)
[18:11] * TMM (~hp@sams-office-nat.tomtomgroup.com) Quit (Quit: Ex-Chat)
[18:12] * marrusl (~mark@209-150-43-182.c3-0.wsd-ubr2.qens-wsd.ny.cable.rcn.com) has joined #ceph
[18:13] * Boltsky (~textual@cpe-198-72-138-106.socal.res.rr.com) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[18:13] * sjustwork (~sam@2607:f298:a:607:89e2:6c6a:f4b8:3977) has joined #ceph
[18:14] * ShaunR (~ShaunR@staff.ndchost.com) Quit ()
[18:16] * odyssey4me (~odyssey4m@165.233.71.2) Quit (Read error: Connection reset by peer)
[18:18] * zoltan (~zoltan@2001:620:20:16:f49b:d96f:edc1:a2c4) Quit (Quit: Leaving)
[18:24] * rotbeard (~redbeard@b2b-94-79-138-170.unitymedia.biz) Quit (Quit: Leaving)
[18:27] * kaizh (~kaizh@128-107-239-236.cisco.com) has joined #ceph
[18:27] * xmltok (~xmltok@cpe-76-90-130-148.socal.res.rr.com) has joined #ceph
[18:29] * xmltok (~xmltok@cpe-76-90-130-148.socal.res.rr.com) Quit ()
[18:31] * fdmanana (~fdmanana@bl9-168-202.dsl.telepac.pt) Quit (Quit: Leaving)
[18:46] * xmltok (~xmltok@cpe-76-90-130-148.socal.res.rr.com) has joined #ceph
[18:46] * hjjg_ (~hg@p3EE3262E.dip0.t-ipconnect.de) Quit (Ping timeout: 480 seconds)
[18:49] * thb (~me@0001bd58.user.oftc.net) Quit (Quit: Leaving.)
[18:49] * xmltok (~xmltok@cpe-76-90-130-148.socal.res.rr.com) Quit (Read error: Connection reset by peer)
[18:55] * nwat (~textual@eduroam-246-150.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[19:00] * fdmanana (~fdmanana@bl9-168-202.dsl.telepac.pt) has joined #ceph
[19:00] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) has joined #ceph
[19:02] * xarses (~andreww@12.164.168.117) Quit (Read error: Operation timed out)
[19:03] * JoeGruher (~JoeGruher@134.134.139.76) has joined #ceph
[19:05] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) has joined #ceph
[19:07] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) has joined #ceph
[19:08] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[19:09] * meis3 (~meise@oglarun.3st.be) Quit (Quit: leaving)
[19:15] * rotbeard (~redbeard@2a02:908:df10:6f00:76f0:6dff:fe3b:994d) has joined #ceph
[19:16] * diegows (~diegows@190.190.5.238) Quit (Ping timeout: 480 seconds)
[19:18] * JeffK (~JeffK@38.99.52.10) has joined #ceph
[19:19] * geekmush (~Adium@cpe-66-68-198-33.rgv.res.rr.com) Quit (Quit: Leaving.)
[19:20] * geekmush (~Adium@cpe-66-68-198-33.rgv.res.rr.com) has joined #ceph
[19:21] * WarrenUsui (~Warren@2607:f298:a:607:c5a4:40bb:68ee:c717) Quit (Ping timeout: 480 seconds)
[19:21] * WarrenUsui (~Warren@2607:f298:a:607:c568:4757:5bf1:b42f) has joined #ceph
[19:21] * warrenSusui (~Warren@2607:f298:a:607:c5a4:40bb:68ee:c717) Quit (Ping timeout: 480 seconds)
[19:21] * neurodrone (~neurodron@static-108-30-171-7.nycmny.fios.verizon.net) Quit (Quit: neurodrone)
[19:21] * warrenSusui (~Warren@2607:f298:a:607:c568:4757:5bf1:b42f) has joined #ceph
[19:25] * markednmbr1 (~markednmb@cpc8-lewi13-2-0-cust979.2-4.cable.virginm.net) has joined #ceph
[19:25] <markednmbr1> hi all
[19:25] <markednmbr1> first off - ceph noob, so be nice ;)
[19:26] <markednmbr1> on my first ceph test, i have created everything and it looked ok. however since a reboot none of the osd's will activate now
[19:26] <markednmbr1> I'm getting the following error
[19:26] <markednmbr1> 2014-03-10 18:20:10.740829 7f20cf6a8780 -1 journal check: ondisk fsid 00000000-0000-0000-0000-000000000000 doesn't match expected 8e9ef9bd-775b-40d0-b3a4-5a05d735ec1f, invalid (someone else's?) journal
[19:27] <markednmbr1> the journals are on LVM volumes on an ssd
[19:31] * dmsimard1 (~Adium@ap03.wireless.co.mtl.iweb.com) has joined #ceph
[19:34] * dmsimard2 (~Adium@ap05.wireless.co.mtl.iweb.com) has joined #ceph
[19:35] * c74d (~c74d@2002:4404:712c:0:281c:caf6:40c0:e857) Quit (Remote host closed the connection)
[19:35] * c74d (~c74d@2002:4404:712c:0:6c2d:cc71:d8f:3e81) has joined #ceph
[19:36] * dmsimard3 (~Adium@108.163.152.66) has joined #ceph
[19:37] <fghaas> markednmbr1: how did you deploy, puppet?
[19:37] * dmsimard (~Adium@108.163.152.2) Quit (Ping timeout: 480 seconds)
[19:37] <markednmbr1> no, just all manually on cli
[19:37] <fghaas> I ask because at least one of the puppet modules out there had a bug that would do exactly that
[19:38] * dmsimard2 (~Adium@ap05.wireless.co.mtl.iweb.com) Quit (Read error: Operation timed out)
[19:38] <fghaas> what it means is that your journal file is either empty or hasn't been initialized as a journal
[19:38] <fghaas> this shouldn't happen with ceph-deploy, though, and I've only seen it in conjunction with that puppet bug, so I may not be of help here
[19:39] <markednmbr1> strange, It is entirely possible I did something wrong, but it seemed ok after I did it..
[19:39] * diegows (~diegows@190.190.5.238) has joined #ceph
[19:40] * ircolle (~Adium@2601:1:8380:2d9:787d:3113:d2c6:7618) Quit (Quit: Leaving.)
[19:40] <markednmbr1> I haven't got anything in ceph.conf for [osd]
[19:41] <markednmbr1> should I have entered something in here after ceph-deploy or should it be saved on disk?
[19:41] * dmsimard1 (~Adium@ap03.wireless.co.mtl.iweb.com) Quit (Ping timeout: 480 seconds)
[19:41] * Siva (~sivat@vpnnat.eglbp.corp.yahoo.com) Quit (Ping timeout: 480 seconds)
[19:49] * dpippenger (~riven@cpe-198-72-157-189.socal.res.rr.com) Quit (Quit: Leaving.)
[19:53] * zerick (~eocrospom@190.114.248.76) Quit (Ping timeout: 480 seconds)
[19:54] * jedanbik (~jedanbik@dhcp152-54-6-141.wireless.europa.renci.org) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[19:54] <fghaas> markednmbr1: nope, ceph-deploy will create your journal and then make a symlink named "journal" in the root of your osd filestore by default
[19:56] <markednmbr1> is there a way to recreate the journal without destroying the data on the osd?
[19:56] <markednmbr1> not that it matters in this instance as its just testing, but for future reference..
[19:57] * reed (~reed@75-101-54-131.dsl.static.sonic.net) has joined #ceph
[19:58] <fghaas> markednmbr1: man ceph-osd, look for --mkjournal
[20:01] * sroy (~sroy@207.96.182.162) Quit (Quit: Quitte)
[20:03] <markednmbr1> Hmm that doesn't seem to work for me
[20:03] <markednmbr1> unless im doing something wrong
[20:03] <markednmbr1> http://pastie.org/8905142
[20:03] * kaizh (~kaizh@128-107-239-236.cisco.com) Quit (Remote host closed the connection)
[20:05] <markednmbr1> there is nothing in /var/lib/ceph/osd at all
[20:05] <dmick> does the journal volume exist behind that sym...oh
[20:05] <dmick> um, is your journal fs mounted?
[20:06] <markednmbr1> its an lvm volume
[20:07] <markednmbr1> it's not mounted
[20:07] <markednmbr1> do you need to mount journal volumes manually?
[20:08] * nwat (~textual@eduroam-246-150.ucsc.edu) has joined #ceph
[20:09] <dmick> sorry, I meant osd volume
[20:09] <dmick> (that should contain a symlink to the journal device)
[20:10] <dmick> osd *fs* (sorry)
[20:10] <markednmbr1> no the device isn't mounted either
[20:10] <markednmbr1> do you need to do that manually
[20:10] <fghaas> well there ought to be a directory named osd-$id within /var/lib/ceph/osd
[20:10] <dmick> the provisioning process needs to arrange for the OSD FS to be availbale for the OSD to strat
[20:11] <fghaas> this all looks like your ceph-deploy only got done about half way
[20:11] <dmick> there's udev gunge to do that if you provision with the right label and partition types, but otherwise, it's up to whoever set up the cluster
[20:11] <dmick> (it wasn't ceph-deploy, I'm betting)
[20:11] * xarses (~andreww@12.164.168.117) has joined #ceph
[20:12] <markednmbr1> I did use ceph-deploy...
[20:12] <markednmbr1> I was following this http://ceph.com/docs/master/rados/deployment/ceph-deploy-new/
[20:12] <markednmbr1> and just kept clicking next ;)
[20:12] * geraintjones (~geraint@222-152-77-45.jetstream.xtra.co.nz) has joined #ceph
[20:13] * shang (~ShangWu@220-135-203-169.HINET-IP.hinet.net) Quit (Quit: Ex-Chat)
[20:14] * kaizh (~kaizh@128-107-239-234.cisco.com) has joined #ceph
[20:14] <dmick> ok. well then they should have been mounted and started.
[20:14] <markednmbr1> it was working when I first ran the commands, but since I have rebooted it hasn't come back up
[20:16] <dmick> sgdisk -l (disk-with-OSD) says?...
[20:16] * sarob (~sarob@c-50-161-65-119.hsd1.ca.comcast.net) Quit (Remote host closed the connection)
[20:17] * sarob (~sarob@2601:9:7080:13a:5df8:efe2:4bf4:7c4f) has joined #ceph
[20:17] * lavi (~lavi@eth-seco21th2-46-193-64-32.wb.wifirst.net) has joined #ceph
[20:19] * jedanbik (~jedanbik@dhcp152-54-6-141.wireless.europa.renci.org) has joined #ceph
[20:19] <markednmbr1> nothing..
[20:19] <markednmbr1> i've manually mounted all the devices in /var/lib/ceph/osd/ceph-X/
[20:20] <markednmbr1> and restarted ceph, and it's not giving me the error anyumore
[20:20] <dmick> sigh, sorry, not -l, -p
[20:20] <markednmbr1> and has created everything
[20:20] <dmick> yeah, so something about the automounting isn't working
[20:20] * BManojlovic (~steki@fo-d-130.180.254.37.targo.rs) has joined #ceph
[20:21] <markednmbr1> http://pastie.org/8905186
[20:21] <dmick> did ceph-deploy create your partitions, or did you?
[20:22] <markednmbr1> I think ceph did, as I just specified the device when I first ran it with ceph-deploy osd prepare node1:sdb:/dev/mapper/journal-osd1
[20:23] <dmick> odd, then. I'd probably look at udev logs, and/or crank up udev debugging. that's what should have mounted the filestore.
[20:24] * dmsimard (~Adium@70.38.0.246) has joined #ceph
[20:25] * sarob (~sarob@2601:9:7080:13a:5df8:efe2:4bf4:7c4f) Quit (Ping timeout: 480 seconds)
[20:25] <markednmbr1> why does udev mount it? shouldn't the ceph init do the osd mounting or an entry in fstab?
[20:25] * flaxy (~afx@78.130.174.164) Quit (Remote host closed the connection)
[20:25] * flaxy (~afx@78.130.174.164) has joined #ceph
[20:28] * dmsimard3 (~Adium@108.163.152.66) Quit (Ping timeout: 480 seconds)
[20:29] <markednmbr1> looks like it was never really working correctly as ceph status gives me the following:
[20:29] <markednmbr1> health HEALTH_WARN 208 pgs stuck inactive; 208 pgs stuck unclean; mds cluster is degraded
[20:31] <markednmbr1> and it just hangs if I try query one of the pgs
[20:32] <dmick> udev mounts it so you can have auto-add from a shelf of spare OSD disks
[20:32] <markednmbr1> Ah right - is there a ceph udev rule or something?
[20:32] <dmick> yep
[20:33] <markednmbr1> ok
[20:33] <markednmbr1> any idea why everything is stuck? the message is
[20:33] <markednmbr1> pg 2.d is stuck unclean since forever, current state creating, last acting []
[20:36] <markednmbr1> right I'm gunna start from scratch as I must have done something wrong here
[20:38] * xarses (~andreww@12.164.168.117) Quit (Ping timeout: 480 seconds)
[20:40] <dmick> are all your osds and monitors running?
[20:41] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) Quit (Read error: Connection reset by peer)
[20:41] * sjm (~sjm@pool-108-53-56-179.nwrknj.fios.verizon.net) has joined #ceph
[20:46] * xarses (~andreww@12.164.168.117) has joined #ceph
[21:07] * danieagle (~Daniel@179.182.144.28) has joined #ceph
[21:08] <markednmbr1> yea everything was running as far as I could tell.
[21:08] <markednmbr1> all commands executing correctly
[21:08] <markednmbr1> i'm going to zap all the disks and prepare again
[21:10] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has joined #ceph
[21:10] * ChanServ sets mode +v andreask
[21:12] * dmsimard (~Adium@70.38.0.246) Quit (Read error: Connection reset by peer)
[21:12] * dmsimard (~Adium@ap03.wireless.co.mtl.iweb.com) has joined #ceph
[21:15] * andreask (~andreask@h081217016175.dyn.cm.kabsi.at) has left #ceph
[21:15] * dmsimard1 (~Adium@108.163.152.2) has joined #ceph
[21:20] * dmsimard (~Adium@ap03.wireless.co.mtl.iweb.com) Quit (Ping timeout: 480 seconds)
[21:25] * JoeGruher (~JoeGruher@134.134.139.76) Quit (Remote host closed the connection)
[21:26] * PerlStalker (~PerlStalk@2620:d3:8000:192::70) Quit (Remote host closed the connection)
[21:27] * jeff-YF (~jeffyf@67.23.117.122) Quit (Quit: jeff-YF)
[21:27] * markednmbr1 (~markednmb@cpc8-lewi13-2-0-cust979.2-4.cable.virginm.net) Quit (Quit: Leaving)
[21:29] * fdmanana (~fdmanana@bl9-168-202.dsl.telepac.pt) Quit (Quit: Leaving)
[21:32] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) has joined #ceph
[21:35] * jedanbik (~jedanbik@dhcp152-54-6-141.wireless.europa.renci.org) Quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz???)
[21:42] * floob (~floob@75-142-107-31.static.mtpk.ca.charter.com) has joined #ceph
[21:43] * Sargun (~sargun@208-106-98-2.static.sonic.net) Quit (Remote host closed the connection)
[21:45] * dpippenger (~riven@66-192-9-78.static.twtelecom.net) has joined #ceph
[21:47] * Cnidus (~cnidus@216.129.126.126) has joined #ceph
[21:47] * Sysadmin88 (~IceChat77@176.254.32.31) has joined #ceph
[21:52] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[21:52] * mtanski (~mtanski@69.193.178.202) has left #ceph
[21:59] * jeff-YF (~jeffyf@216.14.83.26) has joined #ceph
[22:04] <floob> Everything I've read points to "no", but I'll ask anyway, just in case: does Ceph support encrypted replication over a public cluster network?
[22:05] <dwm> floob: I believe CephX provides strong authentication of cluster agents, but I haven't seen mention of over-the-wire encryption.
[22:08] <floob> dwm: I read the Cephx guide, it seems like it solves the client-server problem, but not the server-server / multi-datacenter problem.
[22:08] * flaxy (~afx@78.130.174.164) Quit (Read error: Connection reset by peer)
[22:09] * flaxy (~afx@78.130.174.164) has joined #ceph
[22:12] <floob> I found a reference* to some multi-datacenter features being implemented a few months back, but there aren't a lot of details. [*]: http://ceph.com/releases/v0-72-emperor-released/
[22:12] * i_m (~ivan.miro@nat-5-carp.hcn-strela.ru) Quit (Quit: Leaving.)
[22:12] <dwm> floob: I've not looked into it, but I'd be tempted to investigate opportunistic encryption schemes using IPSec, perhaps using strongSwan?
[22:14] <floob> dwn: I hadn't heard of that, I'll definitely go digging in that direction, too. Thanks.
[22:14] <floob> *dwm
[22:15] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has joined #ceph
[22:15] * MarkN (~nathan@142.208.70.115.static.exetel.com.au) has left #ceph
[22:15] <dwm> I don't know how latency-sensitive the Ceph networking protocols are, bear in mind that encryption of network traffic is likely to have a non-trivial impact on network (and thus cluster) throughput.
[22:16] <dwm> Consider using crypto schemes where you have hardware support, or investigating some of the more efficient elliptic-curve ciphers.
[22:16] <dwm> ... though given recent revelations, you might prefer to use cryptosystems that do not rely on magic numbers supplied by NIST!
[22:19] * flaxy (~afx@78.130.174.164) Quit (Ping timeout: 480 seconds)
[22:19] * JoeGruher (~JoeGruher@jfdmzpr04-ext.jf.intel.com) has joined #ceph
[22:19] <lurbs> If you want to ensure that all of the traffic is encrypted, and would prefer no traffic over leaked unencrypted traffic, check out iptables' policy matching for IPsec.
[22:25] <floob> lurbs: thanks. I'll have to dig there too. I've tried getting into iptables a few times, and only really scratched the surface.
[22:29] * markbby (~Adium@168.94.245.4) has joined #ceph
[22:29] * c74d (~c74d@2002:4404:712c:0:6c2d:cc71:d8f:3e81) Quit (Remote host closed the connection)
[22:29] * c74d (~c74d@2002:4404:712c:0:2cf5:8a4c:b6bc:24a0) has joined #ceph
[22:31] * ircolle (~Adium@2601:1:8380:2d9:8443:2c2b:9af9:d104) has joined #ceph
[22:32] * markbby (~Adium@168.94.245.4) Quit ()
[22:38] * Cataglottism (~Cataglott@dsl-087-195-030-170.solcon.nl) Quit (Quit: My Mac Pro has gone to sleep. ZZZzzz???)
[22:39] * erice (~erice@c-98-245-48-79.hsd1.co.comcast.net) has joined #ceph
[22:45] * erice_ (~erice@c-98-245-48-79.hsd1.co.comcast.net) Quit (Ping timeout: 480 seconds)
[22:47] * mattt (~textual@cpc9-rdng20-2-0-cust565.15-3.cable.virginm.net) Quit (Quit: Computer has gone to sleep.)
[22:48] * sarob (~sarob@2001:4998:effd:600:7132:21be:778d:6f6f) has joined #ceph
[22:49] * jeff-YF_ (~jeffyf@67.23.123.228) has joined #ceph
[22:51] * jeff-YF (~jeffyf@216.14.83.26) Quit (Read error: Operation timed out)
[22:51] * jeff-YF_ is now known as jeff-YF
[22:53] * valeech (~valeech@ip72-205-7-86.dc.dc.cox.net) Quit (Quit: valeech)
[22:56] * japuzzo (~japuzzo@ool-4570886e.dyn.optonline.net) Quit (Quit: Leaving)
[22:58] * valeech (~valeech@ip72-205-7-86.dc.dc.cox.net) has joined #ceph
[23:00] * floob (~floob@75-142-107-31.static.mtpk.ca.charter.com) Quit (Quit: Leaving)
[23:01] * sputnik13 (~sputnik13@wsip-68-105-248-60.sd.sd.cox.net) Quit (Ping timeout: 480 seconds)
[23:01] * valeech (~valeech@ip72-205-7-86.dc.dc.cox.net) Quit ()
[23:04] * jpierre03 (~jpierre03@voyage.prunetwork.fr) Quit (Ping timeout: 480 seconds)
[23:04] * floob (~floob@75-142-107-31.static.mtpk.ca.charter.com) has joined #ceph
[23:11] * xarses (~andreww@12.164.168.117) Quit (Quit: Leaving)
[23:15] * sage (~quassel@2607:f298:a:607:49b8:da33:cd75:9ce3) Quit (Remote host closed the connection)
[23:15] * sage (~quassel@2607:f298:a:607:b498:be93:d4e9:b16) has joined #ceph
[23:15] * ChanServ sets mode +o sage
[23:20] * fghaas (~florian@91-119-229-245.dynamic.xdsl-line.inode.at) Quit (Quit: Leaving.)
[23:22] * alfredodeza (~alfredode@198.206.133.89) has joined #ceph
[23:24] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has joined #ceph
[23:24] * zidarsk8 (~zidar@89-212-142-10.dynamic.t-2.net) has left #ceph
[23:27] * markbby (~Adium@168.94.245.3) has joined #ceph
[23:27] * BillK (~BillK-OFT@124-148-93-148.dyn.iinet.net.au) has joined #ceph
[23:27] * lxo (~aoliva@lxo.user.oftc.net) Quit (Remote host closed the connection)
[23:28] * lxo (~aoliva@lxo.user.oftc.net) has joined #ceph
[23:34] * rendar (~s@95.238.179.204) Quit ()
[23:34] * jpierre03 (~jpierre03@voyage.prunetwork.fr) has joined #ceph
[23:39] * mtanski (~mtanski@69.193.178.202) has joined #ceph
[23:39] * allsystemsarego (~allsystem@188.26.167.156) Quit (Quit: Leaving)
[23:40] * MACscr (~Adium@c-98-214-103-147.hsd1.il.comcast.net) Quit (Quit: Leaving.)
[23:42] * Midnightmyth (~quassel@93-167-84-102-static.dk.customer.tdc.net) Quit (Ping timeout: 480 seconds)
[23:42] * erice (~erice@c-98-245-48-79.hsd1.co.comcast.net) Quit (Remote host closed the connection)
[23:42] * erice (~erice@c-98-245-48-79.hsd1.co.comcast.net) has joined #ceph
[23:43] * nwat (~textual@eduroam-246-150.ucsc.edu) Quit (Quit: My MacBook has gone to sleep. ZZZzzz???)
[23:47] * jcsp (~Adium@0001bf3a.user.oftc.net) Quit (Quit: Leaving.)
[23:49] * mtanski (~mtanski@69.193.178.202) Quit (Quit: mtanski)
[23:51] * AfC (~andrew@2407:7800:400:1011:2ad2:44ff:fe08:a4c) has joined #ceph
[23:52] * jeff-YF (~jeffyf@67.23.123.228) Quit (Quit: jeff-YF)
[23:56] * jeff-YF (~jeffyf@67.23.117.122) has joined #ceph
[23:59] * jeff-YF (~jeffyf@67.23.117.122) Quit ()
[23:59] * dalgaaf (uid15138@id-15138.ealing.irccloud.com) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.