#ceph IRC Log


IRC Log for 2011-07-11

Timestamps are in GMT/BST.

[0:10] * jbd (~jbd@ks305592.kimsufi.com) has joined #ceph
[0:20] * monrad-51468 (~mmk@domitian.tdx.dk) has joined #ceph
[0:43] * jbd (~jbd@ks305592.kimsufi.com) has left #ceph
[1:05] * s15y (~s15y@sac91-2-88-163-166-69.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[1:53] * u3q (~ben@jupiter.tspigot.net) has joined #ceph
[1:55] * s15y (~s15y@sac91-2-88-163-166-69.fbx.proxad.net) has joined #ceph
[4:12] * deksai (~deksai@96-35-100-192.dhcp.bycy.mi.charter.com) has joined #ceph
[4:46] * deksai (~deksai@96-35-100-192.dhcp.bycy.mi.charter.com) Quit (Quit: Leaving)
[5:06] * votz (~votz@pool-72-78-219-167.phlapa.fios.verizon.net) Quit (Quit: Leaving)
[7:19] * votz (~votz@pool-72-78-219-167.phlapa.fios.verizon.net) has joined #ceph
[8:01] * yoshi (~yoshi@p15251-ipngn1601marunouchi.tokyo.ocn.ne.jp) has joined #ceph
[8:16] * s15y (~s15y@sac91-2-88-163-166-69.fbx.proxad.net) Quit (Ping timeout: 480 seconds)
[8:25] * jbd (~jbd@ks305592.kimsufi.com) has joined #ceph
[8:27] * s15y (~s15y@sac91-2-88-163-166-69.fbx.proxad.net) has joined #ceph
[9:00] * nms (martin@sexyba.be) Quit (Remote host closed the connection)
[9:01] * nms (martin@sexyba.be) has joined #ceph
[9:30] * jbd (~jbd@ks305592.kimsufi.com) has left #ceph
[10:16] * jbd (~jbd@ks305592.kimsufi.com) has joined #ceph
[14:31] * mtk (~mtk@ool-182c8e6c.dyn.optonline.net) has joined #ceph
[15:15] * yoshi (~yoshi@p15251-ipngn1601marunouchi.tokyo.ocn.ne.jp) Quit (Remote host closed the connection)
[16:48] * yoshi (~yoshi@KD027091032046.ppp-bb.dion.ne.jp) has joined #ceph
[17:05] * yoshi (~yoshi@KD027091032046.ppp-bb.dion.ne.jp) Quit (Remote host closed the connection)
[17:08] * Tv (~Tv|work@ip-64-111-111-107.dreamhost.com) has joined #ceph
[17:13] * rsharpe (~Adium@ has joined #ceph
[17:32] * rsharpe (~Adium@ Quit (Quit: Leaving.)
[19:02] <yehudasa> stingray: car parking?
[19:06] * joshd (~joshd@ip-64-111-111-107.dreamhost.com) has joined #ceph
[19:12] * sjust (~sam@ip-64-111-111-107.dreamhost.com) Quit (Read error: Connection reset by peer)
[19:36] <bchrisman> sagewk: we're out for all-hand mtg this week...
[19:37] <sagewk> bchrisman: whole week?
[19:40] <bchrisman> heh.. yeah???
[20:08] * cmccabe (~cmccabe@ has joined #ceph
[20:18] <gregaf> Tv: hmm, are you sure autotest includes xfstests? I don't see it in their git repo or your tarball
[20:18] <gregaf> though I might be missing something
[20:18] <Tv> gregaf: hold on
[20:18] <Tv> now why do i remember that :(
[20:19] <Tv> gregaf: sorry if i misled you, can't find it either
[20:19] <Tv> i think that was just on the (infinite) todo list
[20:23] <gregaf> no worries
[20:24] * sjust (~sjust@ip-64-111-111-107.dreamhost.com) has joined #ceph
[20:58] <cmccabe> so
[20:58] <cmccabe> I'm starting to think that httplib is the one doing MIME-encoding on our utf8-dirty metadata
[20:58] <cmccabe> so I really only have two choices for testing what actually happens when you try to store non-utf8 data in s3
[20:59] <cmccabe> use some kind of C program, or monkeypatch httplib
[21:14] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) has joined #ceph
[21:19] <Tv> cmccabe: can you pass it str instead of unicode?
[21:19] * bchrisman (~Adium@c-98-207-207-62.hsd1.ca.comcast.net) Quit (Quit: Leaving.)
[21:19] <Tv> also, there's http://code.google.com/p/httplib2/
[21:20] <cmccabe> I don't see why string vs. unicode string would matter
[21:20] <cmccabe> if it's being "helpful" in one case it probably wants to be helpful in the other
[21:20] <cmccabe> I guess I could try it
[21:21] <Tv> worst case, you talk to a straight up TCP socket
[21:21] <cmccabe> the /usr/bin/s3 command seems to have some way of specifying x-amz-meta headers, so maybe that will work
[21:21] <cmccabe> but I can't even find out who made /usr/bin/s3 or what version of it I have
[21:21] <cmccabe> apparently implementing --version or printing an author name was not important enough to do
[21:21] <cmccabe> and searching for "s3" doesn't get me any hits
[21:21] <cmccabe> for the program
[21:22] <Tv> no calling of /usr/bin/s3 in s3-tests nosetests please
[21:23] <cmccabe> anyway, test_object_set_get_non_utf8_metadata passes a regular string, no u in front
[21:23] <cmccabe> talking to a raw tcp socket is not possible since I would need to re-implement aws authentication
[21:24] <cmccabe> remember it's not HTTPS based
[21:24] <Tv> https is unrelated
[21:24] <Tv> i think you can ask boto to sign things for you, at least with some hackery
[21:25] <Tv> yeah just pass in a http_request it seems
[21:26] <cmccabe> the point is, I can't use http_request because the mangling is happening there
[21:26] <yehudasa> cmccabe: just take a tcpdump of both cases
[21:26] <yehudasa> cmccabe: s3 is from libs3
[21:26] <cmccabe> yehudsasa: yeah, that's a pretty good idea
[21:30] <Tv> is this about test_object_set_get_non_utf8_metadata and friends?
[21:30] <Tv> i'm not sure anymore
[21:30] <Tv> if it is, your httplib theory needs to explain why it behaves differently against aws
[21:32] <cmccabe> tcpdump will answer this
[21:37] <cmccabe> I'm having a lot of trouble interpreting this dump
[21:38] <cmccabe> I thought ss3.amazonaws.com didn't use ssl, but I think this is
[21:38] * lxo (~aoliva@1GLAACP6L.tor-irc.dnsbl.oftc.net) Quit (Remote host closed the connection)
[21:39] <Tv> cmccabe: it can do both
[21:39] <Tv> it's up to the client to decide
[21:39] <Tv> is_secure=False is your friend
[21:40] <cmccabe> tv: yeah
[21:44] <cmccabe> ok, that is a lot more readable
[21:44] <cmccabe> I guess obviously wireshark can't read SSL; that's kind of the point
[21:45] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) Quit (Quit: Leaving)
[21:45] <cmccabe> anyway, it looks like it is being sent over the wire not mime encoded.
[21:45] <cmccabe> so amazon's docs are wrong again, and rgw needs to do that encoding to be compatible
[21:47] <Tv> cmccabe: where are aws docs wrong?
[21:47] <cmccabe> tv: "If some metadata contains unprintable characters, it is not returned. Instead, the "x-amz-missing-meta" header is returned with a value of the number of the unprintable metadata entries. Each name, value pair must conform to US-ASCII when using REST and UTF-8 when using SOAP or browser-based uploads via POST."
[21:48] <Tv> oh where's that?
[21:48] <cmccabe> tv: s3 developer's guide
[21:48] <cmccabe> tv: in reality, you never get x-amz-missing-meta no matter what garbage you send
[21:48] <cmccabe> tv: and sending non-utf8 is cheerfully handled by mime-encoding on the server-side
[21:48] <Tv> yeah i didn't see that in the api reference
[21:48] <Tv> though api ref fails to mention the quoting
[21:48] <cmccabe> search the pdf
[21:49] <cmccabe> the 2006 one which is still the latest they've released
[21:51] <Tv> oh heh, the authentication specifies it's a hmac of utf-8 encoded data ;)
[21:51] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) has joined #ceph
[21:52] <cmccabe> tv: yeah, they more or less try to mandate utf-8 everywhere, which makes sense
[21:52] * lxo (~aoliva@1GLAACQTN.tor-irc.dnsbl.oftc.net) has joined #ceph
[21:52] <Tv> it would, if they enforced it ;)
[21:52] <cmccabe> tv: yeah, I don't understand why they don't just reject invalid utf-8 metadata
[21:53] <cmccabe> tv: they're java programmers, enforcing invariants ought to be second nature to them
[21:53] <cmccabe> tv: probably the docs are just out of date or something
[21:54] <Tv> "it started working, we're done"
[21:54] <cmccabe> stones, glass hosues
[21:54] <cmccabe> houses
[21:55] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) Quit ()
[21:55] <cmccabe> anyway, brb, lunch
[22:15] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) has joined #ceph
[22:21] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) Quit (Quit: This computer has gone to sleep)
[23:03] * rsharpe (~Adium@ has joined #ceph
[23:39] * ghaskins_mobile (~ghaskins_@66-189-113-47.dhcp.oxfr.ma.charter.com) has joined #ceph

These logs were automatically created by CephLogBot on irc.oftc.net using the Java IRC LogBot.