Google presented a talk about the MySQL conference about how they deploy MySQL.
We use the same general technique here at Tailrank.
But they published a pretty interesting hack/trick for fast network IO:
on the target:
nc l p 12345 | tar xvf
no the source:
tar cvf * | nc target 12345
-
Spinn3r News- Cornell Researchers Launch Memetracker Powered by Spinn3r
- Spinn3r Sponsors 2009 International Conference for Weblogs and Social Data Challenge
- Spinn3r 2.3.1
- Spinn3r Hiring Director of Sales
- Feed Update Protocols and SUP
- Spinn3r 2.2.5
- Spinn3r 2.2.3
- Spinn3r 2.2.2 Escapes From the Laboratory
- Spinn3r Hiring Senior Systems Administrator
- Spinn3r 2.2.1 Released
-
Subscribe
-
Flickr
More Photos
April 28, 2007 at 6:57 am
I think you’re missing a couple dashes in your commands. Specifying a dash for the filename in tar means stdin/stdout, depending on operation. So, it’d be:
nc l p 12345 | tar xvf -
tar cvf - * | nc target 12345
You can combine things using ssh:
tar cBf - * | (ssh target; tar xBf -)
April 29, 2007 at 4:20 pm
Well that looks familiar. I have scripts using that trick from ~2001 or so. I wonder if I remembered to put that in the book… Hm.
April 30, 2007 at 6:43 am
Here’s right from Google’s presentation. 12345 being generic port# obviously
On the target:
– Start nc listening for incoming data
nc l p 12345 | tar xvf •
On the source:
– Push tar’ed files to the target
tar cvf * | nc target 12345