Friday, 5 May 2017

Lines, radios, and cables - oh my

Spread Networks blew a lazy few hundred million dollars on a white elephant straighter optical fibre between Chicago and New York. Not all traders were wise enough to dodge the Spread Networks bullet with the most famous customer spending an unjustifiable inordinate amount being Getco. Microwave had been on the same link for over fifty years, was faster, and already used for trading.

Don't make the same mistake as Spread. Be careful with your link choices and your cable choices.

Look at these cables. Decide the order of speed of propagation of signal in them. Many traders, but not so many engineers, may be surprised:

Basic cabling test: put these in order from slowest to fastest
(click to enlarge)

The correct order from slowest to fastest, by the velocity of propagation, is d, a, c, b, then e. There is faster though. If you're geek enough, like me, to get a kick out of this kind of thing you may find this interesting. Most people would prefer to meander elsewhere I suspect. I'm not the guy you want to invite to your dinner party ;-)

Latency misconceptions


Even savvy traders, such as Getco, do make mistakes and invest millions of dollars inappropriately in the wrong communication technologies. Don’t do that.

Latency may be worth millions of dollars to your trade, but capital and recurrent expenditures may give you pause as you toss around modern HFT technology and potential ROIs. Tech can be expensive. You’d better understand it well before choosing your preferred cost and profile. Let’s have a look at some of the poorly understood and interesting, to me, misunderstandings and developments that may be important to both your latency critical and latency sensitive trading. Let’s meander through some of the points.

Is fibre transmission faster than transmission using electrical wires?

The answer is: it depends.

Is radio frequency transmission always faster than fibre?

The answer is: it depends.

The new low earth orbit (LEO) satellite service in pre-sales from LeoSat Enterprise LLC has reportedly snared a high speed trading customer. Could LeoSat really be faster than terrestrial
communication?


The answer is: it depends (but unlikely).

Back in the day, when Getco released some S1’s & S4’s, there was a bit of trading community comment regarding notes in the accounts where it was disclosed that millions of dollars had been spent on Spread Networks fibre capacity between Chicago and New York,
“Colocation and data line expenses increased $18.9 million (52.0%) to $55.2 million in 2010 from $36.3 million in 2009 primarily due to the introduction of Spread Networks, which is a fiber optic line that transmits exchange and market data between Chicago and New York, and the build out of GETCO’s Asia-Pacific colocations and data lines.” [Knight Holdco, Inc., SEC S-4, 12 Feb 2013, page 227].  
Investing in Spread Networks was wasted money. Microwave links are faster and were already being used on that route. In fact, the first microwave link was built in 1949 for that route.
September 1949 Long Lines publication regarding New York to Chicago microwave link
(click to enlarge)
Later, poor old Getco had their traders’ frustrations aired in public with the disclosure of an internal complaint regarding their internal microwave network being higher latency than a third party network available for use. I expect that was either the McKay Bros / Quincy Data or Tradeworx people. They do good work:

McKay Bros round trip microwave latency. Optical fibre is ~12ms on same path.
(source: click image to enlarge)
I use Getco as an example here not because they are incompetent, but rather because they are very good at what they do. Even Getco, now KCG, now Virtu, as good as they are, had missteps in low latency path development.

Wired 2012. Not so secret, hey Michael Lewis?
If you haven’t read Michael Lewis’ Flash Boys, and you really shouldn’t, you may have missed the low latency narrative centred around Spread Networks’ fibre roll-out that stitched together the book. The literary device used to end the book was the hook of a tower hosting a microwave network. This ending was left as evil hanging in the air like a brick doesn’t. To me, such narrative abuse represents some very poor journalism. Such RF links and vendors offering them had been widely discussed, such as Wired (2012) and the Chicago Tribune (2012). They had weighed in on the microwave discussion publishing vendors names and even prices. This is a snippet from the Chicago Tribune in 2012, years earlier than Flash Boys,
"He [Benti] said the microwave network starts at 350 E. Cermak, ends at another telecom hotel at 165 Halsey St. in Newark, N.J., and went live in the fourth quarter of 2009."
It was hardly a big secret and I found the presentation in Flash Boys somewhat scandalous. Barksdale and Clark, who Lewis had written a book about, “The New New Thing”, are investors in Spread Networks and IEX. They remain friends of Lewis. That looks material to Flash Boys objectivity, or lack thereof. Lewis marketed BS to help his friends.

Latency matters. Latency can be expensive. Latency technology has risks. Let’s expose some latency matters that matter.

Trading at the speed of light


Let’s meander through a little physics and then some of the histories of some communication links.

The speed of light is 299,792.458 km/s, near enough to 300,000 km/s which is how I usually round it off. This is the speed of light in a vacuum and the newish way that we define time. Light’s speed is commonly referred to as ‘c’. When you force light, which is just a form of RF, into a medium, such as a fibre, for light, or a wire, for RF, it goes a bit slower. You’ll have to remember that electrical transmission is different to photonic transmission but related.

The speed of light in a standard optical fibre, either single mode fibre, or multi mode fibre, is around two-thirds of the speed of light in a vacuum. This is normally written as 0.66c.

The atmosphere isn’t a vacuum, thankfully for life on earth, but it doesn’t slow RF, including light, much at all. It’s close enough to c that we don’t bother with a discount and just say it’s 1c.

This is why microwaves make a big difference. If you could use point to point transmission for the roughly 1200 kilometres from Chicago to New York you’d get roughly 6 milliseconds for light in standard fibre and 4 milliseconds for direct RF transmission. Two million nanoseconds of difference is quite a big difference to a trader.

Spread Networks dug very straight lines and managed to beat other fibre networks to achieve the lowest fibre latency known on that link.  Spread’s latency was indeed around 6ms one way, as expected.  It’s a shame they didn’t fully appreciate the benefits of microwave comms on that link before they started digging. Let’s stick to just the tech for now. Here is the start of a table:

Medium Speed of transmission
Vacuum 1c
Atmosphere ~1c
Twisted pair ~0.67c
Standard fibre ~0.66c

That’s pretty rough and I’ve taken a few liberties which I’ll explain later. A very important and interesting thing about electrical transmission in wire is that the construction of the wire, and, perhaps even more importantly, the insulation, matters. Not just a bit, but a lot.

LMR-1700 coaxial cable specs.Note the Velocity of Propagation is 0.89c
(click to enlarge)
If the “wire” was a coaxial cable, then the RF would enjoy travelling along the outside of those wires’ surfaces and burn rubber to achieve up to 0.89c [LMR-1700 low loss coax – Foam PE and 0.87c with Commscope 875 coax also with Foam PE].

Remarkably, older coaxial cables undersea cables used from around the 1930s might have been faster than some modern fibre cables. Not many people understand that. Then again if you chose Neoprene as the dielectric in your wire cable, as earlier cables did, you’d only chug along at around 0.45c. The dielectric performance of the wire limits the speed. A high dielectric constant in your wire is bad news for latency. In the nineteenth century, most cables used Gutta-percha compounds which had dielectrics in the range of 2.4 to 3.4 with over 4 when wet, likely resulting in speeds significantly less than 0.66c. In the 1930s the coaxial submarine cables around the world started using polyethylene which has a typical dielectric constant
Coaxial cable
of 2.26 giving a speed of around 0.66c. However, the construction matters a lot. A foam polyethylene has a dielectric constant of around 1.55 resulting in typical speeds of around 0.8c.

Open ladder line cable
Old tech can be fun. You can get more than 0.95c out of a simple open wire ladder line. 0.95c to 0.99c is the typical range for open ladder lines. You might remember an open wire ladder line if you cast your mind back to that really old school two parallel wire TV antenna cable with rectangular cut outs in the polyethylene webbing every inch or so. Who’d have thunk it? Ancient technology faster than fibre! Details matter.

Printed circuit board (PCB) design is both a science and an art. Standard PCB layers use a medium called FR4, basically fibreglass, which is probably the most common PCB filler. PCB transmission with FR4 is positively glacial with 0.5c being typical. Various other layers, such as Rogers, are used for high-speed channel and RF design which has different properties again and is typically faster for latency too.

Let’s look at a revised table:

Medium Speed of transmission
Vacuum 1c
Atmosphere ~1c
Open wire ladder ~0.95c
Coaxial cable ~0.8c
Twisted pair ~0.67c
Standard fibre ~0.66c
PCB FR4 ~0.5c

Now you can probably imagine building a twisted pair cable that is a bit rounder, more like coax, and not so flat, less of a PCB, and that cable may be a little faster. So, the revised cable might be faster than light over fibre. Again, details matter.

There are standards for CAT twisted pair cables. Those standards also specify minimum propagation speeds and variations within the cable. For example, here is the standard specification for Cat-6 cable:


The velocities are minimums, so don’t panic yet about the 0.585c to 0.621c. If I look at the specifications for a couple of real world cables I see Draka SuperCat 5/5E at 0.64c, and Prysman M@XLAN cat 5E/6/6A cables claim 0.67c. These are specification claims, not guarantees. Seimon reports in their cable guide,
“NVP varies according to the dielectric materials used in the cable and is expressed as a percentage of the speed of light. For example, most category 5 polyethylene (FRPE) constructions have NVP ranges from 0.65c to 0.70c... Teflon (FEP) cable constructions range from 0.69c to 0.73c, whereas cables made of PVC are in the 0.60c to 0.64c range.”
Those electrons must just find Teflon just as slippery as we do.

The other notable thing in the cable specifications is the maximum delay skew. That refers to the fact that the different wires in the cables may propagate faster or slower. In an old school coaxial submarine cable the core wires could be 5% shorter than the outer wires. That is a big deal. In a twisted pair CAT 6 the 45ns skew per 100m could be around 8% variation within the wire. This can matter a good deal as you may only be as fast as your slowest bit producing bit.

Can you quite believe you are still reading about cables and propagation? This is the kind of detail a good trader may have to worry about.

My friends at Metamako measured some common fibre and wire cables using their latency measuring MetaApp. They found the copper cables they tested were indeed faster than the fibre ones by just a bit. I’ve reproduced Metamako’s chart below with permission:
Metamako cable comparison "Copper is faster than fibre!"

This chart comes from the following data:


Here, copper is faster than fibre.  The direct-attach copper cables come in at 4.60ns per metre, single mode fibre at 4.95ns per metre with multi-mode fibre at 4.98 ns per metre. You always have to be careful looking at this as it is not just about transmission but also about the latency cost of amplifying, cleaning, and propagating the signal. Notably, the fibre has a little more endpoint overhead as you can see a larger constant in the fibre equations’ fits.

As you now know, cables can vary a great deal, so caveat emptor.

Another note about cables is that often longer copper interfaced cables in the data centre aren’t really copper but active optical cables (AOC). Such cables transmute the electrical signal into optical and back to electrical (EOE) as part of the cable to improve range. The constants, especially from media changes, can matter in these equations. For example, with 10G Ethernet over CAT-6, you might use a nice Teflon cable and expect some fast propagation. You will be disappointed to learn the 10G twisted pair codec is really twisted and likely to cost you microseconds, yes, thousands of nanoseconds, before you even get onto the cable. Whilst the rise time of a 10G laser in an SFP+ may be less than 30 picoseconds, organising the rise time from the electrical signal takes some gymnastics, even if quite a bit less than 10G twisted pair. A fast cable, or plane, will not always help if your boarding procedures are slow.

There are some more obscure and exciting cables, such as fewer mode fibre, we’ll save for later.

A little comms history


Let’s segue and meander through a bit of history.

Did you know Great Britain’s Pound is commonly called “cable” in trading and financial circles?

This is because when that first cross-Atlantic telegraph cable briefly sprang into life in 1858, information sped up. An obvious and important use was for trading and financial information, hence the name for the US Dollar to Great British Pound cross rate became colloquially named after its primary Atlantic transmission medium. Cable underscores the importance of cable.


When did these trade latency wars start? Perhaps thousands of years ago but certainly hundreds of years ago. There are records relating to coffee merchants in Africa and the Middle East suggesting a trader knowing quickly about production in Africa could make significant profits in the Middle East. Kipp Rogers pointed me to a letter from a silk merchant from around 1066 worrying about time being wasted waiting for such tradeable information,
“The price in Ramle of the Cyprus silk, which I carry with me, is 2 dinars per little pound. Please inform me of its price and advise me whether I should sell it here or carry it with me to you in Misr (Fustat), in case it is fetching a good price there. By God, answer me quickly, I have no other business here in Ramle except awaiting answers to my letters… I need not stress the urgency of a reply concerning the price of silk”
Latency trades are no “New New Thing.

You’ve probably heard the stories of Rothschild’s consol trade where learning about Waterloo the information was transmitted, most likely by fast boats rather than by the pigeons he is famous for using, to London earning a considerable profit. Reuter’s empire was started by being at the crossroads of information flow to participate and speed up information flows. Alexandre Laumonier pointed out to me the old semaphore and light signalling used by the French as an early optical network. It’s also fun to know there were various frauds, delays, embeddings in early semaphore and telegraph networks with profit motives, even making it into the tale of “The Count of Monte Christo.Chappe’s optical telegraph in France covered 556 stations with 4,800 kilometres of 1c transmission media, air between the stations, from 1792.

Speaking of Chappe's optical telegraph, you may find it intriguing that even in the 1830's stock market speculators were abusing communications for profit,
"On another topic, and like Internet outstripping the lawmakers, optical telegraph asked for new laws and regulations: a fraud related to the introduction, into regular messages, of information about the stock market, was discovered in 1836. The telegraph operators involved were using steganography (a specific pattern of errors added to the message) to transmit the stock market data to Bordeaux. In 1837, they were tried but acquitted, having not violated any explicit law. The criminals were one step ahead."
Many people argue that High Frequency Trading (HFT) is a new phenomenon, perhaps as little as a decade in age. Some argue it goes back to the 1980s. The wise Kipp Rogers also passed on a nice book reference to me which noted HFT, in the modern sense, from 1964’s Bankers Monthly, Volume 81, page 49,
“This is an important aspect of bank stocks and leads us to a clearer view of the market. To begin, let’s define a broad line of bank stock house as one range. There are few professionals who will insist that high-frequency trading occurs in more than 20 bank stock names.” 
The use in the text quite clearly talks about it in a style that suggests common usage so perhaps the term is decades older? HFT is not a “New New Thing.

Indeed, there is not much new under the sun, even in clichés. The power of compound interest was argued in cuneiform some 5,000 years ago. The code of Hammurabi dealt with trade, liability, amongst many other things, over 3,800 years ago. Your friendly Ancient Greek philosopher, Thales was challenged to show how philosophy could be practical in a financial sense, so he made money in times BC, by using options on presses to leverage olive forecasts and cornered the market. Ancient Rome used corporate structures.

Just as HFT is probably older than you and I think, history shows the importance of latency is also not a “New New Thing.

Retransmission


One of the problems with the old semaphores and telegraphs is that humans were used as repeaters. The early telegraph couldn’t cross the US continent with its electrics and thus people rekeyed the messages. Semaphore networks are optical and transmit at the speed of light but the onboarding, off-boarding, and retransmission of messages relies on people, flags, and the like. Such retransmissions were not measured in picoseconds.

This is also an issue for modern microwave networks. Lower cost microwave or millimetre wave devices often have tens or hundreds of microseconds in their onboarding and retransmission latencies. For much of the world wasting a few microseconds is not so much of a big deal. The telecom carriers are usually more concerned with bandwidth as their optimisation point.

The very best Chicago to New York links have single digit microseconds differences between them through aggressive path and device optimisations. So the issue of retransmission latency and onboarding is a large issue. One way of cutting down latency, or to make devices simpler, is to talk to the device with a signal it understands to cut out any unnecessary conversions. This led to radio over fibre (RoF) where the RF signal is represented directly in the fibre to feed microwaves. A significant development more relevant to the Chicago – New York and London – Frankfurt links was the development of clever repeaters that analyse the signal and minimally process the signal if it is of sufficient quality rather than requiring a full digital cleansing, or clock data recovery (CDR).  Such repeating takes nanoseconds instead of microseconds. Most microwave traders now use such microwave repeaters.

The first trans-Atlantic telegraph cable in 1858 was a stupendously expensive and brave undertaking that only briefly worked. Cables had been used across water before, with the English Channel being crossed in 1851, but nothing quite so ambitious as a whole ocean. The Atlantic cable briefly worked thanks to sensitive receivers rather than by an understanding of amplification or repeating. That came later.  Brave souls, newer cables, amplification, and repeating drove improvements and commerce to an ever increasing frenzy. Messages were very expensive to send, but the financial world became a virtually instant world to onlooking humans in the 1850s. The path for the rise of the machines was laid.

Satellites


Geostationary communication satellites came to enable anywhere to anywhere communication covering the entire planet. Telephone systems and faxes were hooked up. If you are old like me and have talked on an old telephone link you’ll remember the satellites’ biggest problem, the nasty delays inherent to the lines. Hearing your own voice delayed, or just an awkwardly pausing conversation would drive you nuts. Latency was the issue.

Geostationary orbit is a high orbit. A really, really high orbit. The circumference of the world is about 40,000 kilometres. Using 2πr you can work out the distance from the centre of the earth to the surface is about 6,400 kilometres. A geostationary orbit is 42,164 kilometres from the centre of the earth. Over six times the earth radius. Pause and visualise that for a second. A tiny, little satellite dot far from Earth. Several Earths away. That’s a long way.  So sending a signal to a satellite and receiving it somewhere else is nearly the same as going around the Earth’s equator twice!  That geostationary 72,000 km journey, there and back again in Hobbit speak, at the speed of light, is around 240 milliseconds. Add some processing overhead and you get a very annoying delay. Geo-stationery satellites suck for latency. Don’t use satellites for trading. That is why we have a bunch of spaghetti surrounding the earth in the guise of under-sea cables. Latency begone.

Current submarine cabling
(click to enlarge)


Microwave


Microwave has been around longer than many people think. A microwave link was put over the English Channel in 1931.  Today’s HFTs are fighting over space at Richborough to make straighter lines with taller towers for less repetition over similar grounds. The first Chicago to New York link was created with 34 jumps in September 1949, just beating Spread Network’s straighter fibre by some 60 years and two milliseconds.

President Truman made a USA coast to coast microwave TV transmission in September 1951 after it was opened for telephone use in August.

Microwave is not a new thing and don’t let Michael Lewis lead you astray into thinking otherwise.

In my homeland of Tasmania, some amateurs set a record using standard astronomy telescopes on a couple of mountains with over 100 miles between them to modulate a voice call.

This example shows that whilst light transmission, including lasers, typically have less of a range than microwave, it doesn’t always have to be that way if you want to get creative. Microwave bandwidth and distances are continually improving in leaps and bounds.

RF, such as microwave does not have to be hideously expensive. My Toronto to NY link regular old telco cable was about $15k rent per month from memory of the interlisted arb. With microwave you could buy a couple of end points for your link and then you don’t have to pay the recurring costs for cables. However, the towers and real estate become an expensive proposition which grows as the links get longer and up goes the repeater count. Lots of HFTs are fighting over similar paths and towers and there is a certain element of land and license grabbing that takes place. Alexandre Laumonier, via his blog SniperInMahwah, has been documenting such links in Europe and the recent battle to get a couple of large towers approved in Richborough in the UK for the channel crossing. It’s an expensive game when you want to build large towers.

The actual microwave RF bit is expensive but it’s not outrageous. The real-estate access and towers can be very expensive. All the HFTs have been knocking each other around with one upmanship to gain an edge. Public spectrum and council records, such as those used by Sniper, make it difficult to use shell games to hide your capabilities. In that spirit, a recently announced consortium of sorts is joining forces to create a “Go West” project to share the burden, as they know they’ll just compete each other out to create very similar links. That makes a good deal of sense as the technical cost is nearly out of control with such networks due to the land and towers. IMC has invested in McKay Bros to facilitate improvements to their networks for the benefit of all traders. Tower Research has joined them. KCG and Jump Trading also work together as New Line Networks. These joint venture approaches are sensible cry outs to the gods of cost control. HFTs are realising that being fastest is not so good if you can’t afford a trade’s transaction cost, including your depreciation.

Radio is light is radio


Marconi was obsessed with crossing the Atlantic with radio. He succeeded in a bit of a scary way. He basically built a huge amplifier than generated enough of a current, or spark, to bludgeon his way across the Atlantic with brute force.
Marconi using a kite to lift his antenna to 150m for the first Atlantic transmission in 1901

Click. Kaboom.

What frequency was it?

All of them!

Well, pretty much. Perhaps around 850kHz. That spark gap transmitter was quite quickly replaced with more nuanced hardware so that different people could use different frequencies and thus the planet was not just restricted to one giant broadcaster. We then found that certain frequencies, 2-70MHz, the high frequency or HF band, would bounce around the world thanks to the Ionosphere acting as a bit of a trampoline, sometimes, to those frequencies. Shortwave radio is not so popular anymore but still active, even for number stations.

Microwave and millimetre radio is a bit of a misnomer. Microwave, in the normal literature, actually covers wavelengths from 300MHz to 300GHz, or wavelengths from 1 millimetre to 1 metre. Millimetre bands are part of the microwave spectrum and do indeed have wavelengths of millimetres. Not having micrometer wavelengths with microwaves I find a little interesting. Micrometre wavelengths are part of the infrared. Typical microwave networks are in the 2GHz to 7GHz range. 60GHz is a popular, usually free, millimetre wave network. It is free as the atmosphere kicks it around and limits its usefulness. Many countries have light weight regulations for 80GHz links so you can use them more easily. There is much for the trader to choose from.

Light is radio. Radio is light. The wavelength for your standard data centre fibre light over MMF is 850nm or ~350 THz. In the datacentre, 1310 nm is typically used. 1550 nm is often used for longer distance links thanks to its kind transmission properties in long strands of fibre. Note that visible light is usually considered to range from violet at 380nm/789THz to red’s 620nm/400THz. The common data centre light borders those visible frequencies. When we put different colours of light, or wavelengths, onto a single fibre, we call it Wave Division Multiplexing (WDM) which is a complicated way of saying a pretty rainbow.

We have standards for the colours, sometimes called channels, so we can talk to each other thanks to the International Telecommunication Union. We mix those colours up and separate them out after to make better use of the holes we dig in the ground or sea. If the colours are close together we call it Dense WDM (DWDM) and when the colours have a bit more space and there are fewer of them it is called Coarse WDM (CWDM). Fancy names for pretty simple stuff. International trading is powered by rainbows, literally.

Often the photo-sensor receivers are wide-ranging enough to allow just about any frequency to trigger them which can make your network design a little easier.  We can often interchange short run MMF and SMF cables without noticing too much as they mainly make a difference in the large runs or over n-way splits. SMF cables used to be expensive cables but they aren’t too different in cost to MMF today in the volume a trader may buy them.

Erbium doped cables take a little light injection and reinvigorate the existing light signal as it travels which is pretty clever. There is no real latency cost here if you consider the erbium-doped length as part of the cable. You need a bit of distance in the cable doping so this is a slow way of doing amplification if you only have short cables, such as in a trading co-lo facility. For short distances, you’ll be better doing OEO which is not so different to the EOE we talked about with AOC cables.

Faster fibre


There has been some interesting work on making much faster fibre cables. The idea had its seed in thinking about point to point laser systems, often called free space optics, that have been used for links, including for HFT in New Jersey. Imagine you do your lasering underground. Carefully add some mirrors to bounce the lasers around. That is not too far from the concept of a Hollow Core Fibre (HCF) or Few-Mode Fibre (FMF). HCF speeds are around 0.997c. Pretty good, no? So why aren’t they everywhere?

Cost has been an issue. I looked recently and cables were about $500 a metre. Yikes! Perhaps HCF cables are cheaper now or in bulk. HCF repeating is an issue as the signal dissipates pretty quickly. That is, the mirror bouncy wouncy timey wimey thing, to paraphrase The Doctor, is not so super efficient. Attenuation kills. We are used to having big distances between our fibre repeaters in modern times. Strangely enough though, the HCF repeater requirements are not too different to the old coaxial cable requirements in terms of spacing. Hmmm, perhaps expensive long distance HCF is really possible for a trader if we go back to the future? A demonstration a couple of years ago changed this thinking when a greater than 1000km repeating FMF cable was demonstrated in Europe. Perhaps we’ll see Spread Networks replace their Chicago to New York link’s SMF with HCF?

Medium Speed of transmission
Vacuum 1c
Atmosphere ~1c
rf inc laser ~1c
Hollow-core fibre ~0.997c
Open wire ladder ~0.95-0.99c
Coaxial cable ~0.45-0.89c
Twisted pair ~0.58-0.75c
Standard fibre ~0.66c
PCB FR4 ~0.5c

HFT radio tuning


A few years ago now, the small HFT I founded bought a couple of the original Ettus Research FPGA GNU Radio boxes to play with. We got a little RF signal to go a short distance in the room. Well, they were sitting on the same workbench. Digital FPGA pin in to digital FPGA pin out was 880ns on the oscilloscope. That’s pretty fast. The experiment was to see what kind of overhead the RF stack, including the IF, encoding, MAC, etc, was causing. This experiment showed that with such modern software defined radio (SDR) this kind of RF comms hackery has become wide open to all types and sizes of trading firms.

Why doesn’t an HFT just use a HAM radio to send a signal across the Atlantic to compete? Well, maybe they are. If HFTs are it definitely requires some custom thinking as commodity appliances for this do not exist with the right characteristics. The MIL-spec stuff, say for non-satellite warship communication, may use HF radio but the packets can take seconds to get through. Ouch. That is slow. Why is it so slow?

Email on warships with HF is slow because the MIL-spec packets are heavily encoded with error correction and are spread out over time to handle disturbances. Now Marconi didn’t do this. His brute force grunt was sent at 1c over the horizon with little processing overhead. Click. Kaboom. It may be possible to spatially encode a signal instead of doing the redundancy over time to instantly deliver small messages from continent to continent that are leading edge triggered. HF Multiple-In-Multiple-Out (MIMO) may also be a thing for those purposes. Just as you have little groups of MIMO antennae with centimetre, or so, spacing on the back of your wi-fi router, HF MIMO can have groups of antennae doing their thing. However, even though the research I’ve seen looks promising, their encoding was still slower than a terrestrial equivalent.  One experiment was going from Europe to the Canary Islands but even though the net result for the experiment is encouraging, it was still slower than cable speed due to those pesky encoding and hardware overheads. Such speed was not the point in that particular case though. Just getting HF MIMO working is quite a feat. There is much potential to explore this area even if HF MIMO has somewhat huge spacing for the antennae. Awkwardly, spacing for HF MIMO antennas is not measured in centimetres but hundreds or thousands of metres. The antennas don’t fit in my workshop but they may fit in your trading farm down in Cornwall.

Another RF alternative is to use line of sight with balloons (google's loon) or planes (facebook and google). This is not new. The height of balloons was not just used in the American Civil War, but a US company used cheapish balloons for making small responders that could help in tracking trucks and other freight things in the US, mainly in the South. To keep costs under control, you got a reward if you found a fallen balloon, read the plaque, and sent it back to the company for your reward. That way they kept recycling RF stations. That same company was also awarded a contract for enlarging the RF footprint in Iraq for the DoD via balloons. Google’s RF balloon trials in New Zealand have been working well. RF balloon comms are no longer a "New, New Thing."

Before I knew all of this, in the dark annals of history, I was interested in looking at the Toronto, Chicago, NJ triangle to see what height might be practical for direct line of sight. The Toronto to New York distance is about 800km. For that distance, a platform would have to be at around 12,500 metres at each end to see each other.  Not so different if you just put a balloon in the middle. Youtube tells me this is clearly possible and not so high if consider all of the high school kids sending weather balloons 100,000 feet high to get pretty pictures and videos of the curvature of the Earth. If The Register can send their Lego man up in a paper airplane to such heights, surely an HFT can do something cute too?

The Register's Paper Aircraft Released Into Space (PARIS)

It is also worth considering HF radio bouncing around the ionosphere. A relatively small transmitter can cover the entire planet. The ionosphere is wide ranging in height of bounce. For simplicity, let’s assume it is at 60km and plug it into a bouncy equation for a long link and the total distance variation is surprisingly small. That is, the HF bouncing around from NY to Tokyo doesn’t add that much onto the total distance due to the shallow angles involved.  If you can find a way to encode the signal sufficiently well, your trading latency could be on a winner.

For some years there has been talk of using balloons across the Atlantic for trading. Hobbyists with model airplanes have flown the distance. Maybe you could use a continuous line of UAVs to act as relays? Now that would be a fun project. An HCF undersea cable seems more practical.

LEO Satellites


There was an article in the Wall Street Journal about LeoSat recruiting HFTs for low latency links. The WSJ reported one HFT taker. We saw previously that height is an issue for geo-stationery satellites as the latency is a killer. So how low is low for LEO? Is the height a latency killer? The company is planning on laser-based comms between the satellites but you still have to get up there. Low cannot be too low for satellites as otherwise there is a bit of pull and drag that sucks them into the atmosphere to a fiery death. LEO orbits start at 160km, might be 300km, but really need to be 600km, or higher, to last a while. That is, it is usually better to have a bit more height and live longer. O3b medium orbit satellites sit at 8,000km. Iridium LEO satellites are at 780 km.

The WSJ article reported LeoSat could do Tokyo to NY in less than 130ms which LeoSat claimed was twice as fast as existing 260ms links. This claim is a little hollow as publicly known Chicago – Tokyo links are already similar in speed to that quoted by LeoSat. Hibernia is offering JPX in Tokyo to CME at Aurora, Chicago as a link at 121.190 ms and we know Chicago to NY is just under 4ms with current offerings such as McKay Bros. 130ms from LeoSat is already not competitive. The article quoted the company as saying satellite to ground latency was 20ms. It’s not clear if that is one way or a round trip for 600km of light speed equivalent. It’s not fast. Low orbit, not so low latency in this case, yet.

Neutrinos and long waves


I hope this has provided some of the colour to the thoughts a trader may have about trading links.

I’ll leave with one further thought. Neutrinos. Hold up your hand. You have hundreds of billions of neutrinos travelling through it each second. There are around 65 billion solar neutrinos passing through matter per square centimetre on Earth. Trillions are passing through your entire body. Near the south pole, there is a cubic kilometre of clearish ice with special sensors hot drilled in. They lie in wait for Neutrinos travelling through the Earth from the North Pole. Those neutrinos occasionally, very very rarely, bump into something and provide a little blue flash.
IceCube: South Pole Neutrino Observatory

A trader might think, why go around the Earth or its crust when you can go through it? Nice.

Remember the fuss that started in Sep 2011 regarding neutrinos travelling faster than light as part of the European OPERA experiment?  It was thrown out to the community to solve the puzzle. Eventually, it was figured to be a measurement error. To me, the interesting part was that someone was firing Neutrinos from Geneva, Switzerland to Gran Sasso in Italy, through the planet, and detecting them! Neutrino communications is a thing already. You need to send an awful lot to get a lucky hit and thus a message length would be short and the time would be probabilistically long, but you gotta start somewhere. Don’t let the detector’s required 300,000 bricks weighing 8.3kg per brick daunt you. What's 300~400GeV between friends? Who wants to build and improve on a few tonnes of Neutrino detection for HFT?

Submarines can use long waves for water penetrating RF comms. Slow packets with big waves. There are patent papers for turning a whole submarine into a Neutrino detector for comms or navigation as an alternative to long waves. Would it work? Seems very unlikely but, tantalisingly, not completely crazy. A few hundred tonnes would not be a problem for a submariner. Such answers are beyond my pay grade but an HFT has gotta ask.

What about ground penetrating long waves? Long waves are slow as they are very long in metres. You have to wait a long time for your bits. Though I do remember when sub-wavelength imaging was thought to be “proven” to be impossible. Super-resolution imaging came along despite rigorous math suggesting ye olde wavelength limiting thingamabobs prevented us diving deeper. We can now see molecules and atoms inside cells by thinking a little outside that wavelength limiting box. That is, in a short while, the impossible became possible. The neural community has weathered two large winters of over a decade each to survive to be a bright deep learning star doing the seemingly impossible despite what Minksy and Papert had you believe in the 1960s. Scientific winters sometimes pass. So, you never know. Perhaps long waves that hug the earth’s curves and penetrate water and soil can go do something sub-wavelength for signalling the British Pound is a buy with one or two bits of secret signal and a new “cable” for cable may be born? Maybe some kind of neutrino or neutrino-like particle can be practically enabled. There is a good movie in there somewhere for Matt Damon to follow up on.

Final word


I’m not holding my breath for the “Go Through” consortium, or cartel, to replace today’s “Go West” venture. That said, I’d be surprised but not shocked. Once Musk gets his Mars trading outpost functioning I hope we don’t repeat the mistakes of the past and build too much duplicate infrastructure for trading Martian Renimbi against the Earth’s Rupiah.

Back in our real world: UAV based comms, hollow core fibres, and HF based HFT low latency signalling may be happening whether you like it or not. Learn from Getco and don’t buy into the “New New Thing” that is just another Spread Networks. Be aware and beware.

--Matt.

[Update: An addendum with a little more on cables for the curiously curious, "Oh my - more lines, radios, and cables"]

2 comments:

  1. In 2012 FermiLab used neutrino's to send an astonishing 0.1 bit per second signal using neutrino's:
    https://www.technologyreview.com/s/427221/first-digital-message-sent-using-neutrinos/

    Anyway, thanks for the great article!

    ReplyDelete