Jump to content
Sign in to follow this  
You need to play a total of 5 battles to post in this section.
1An0maly1

Cryptography in World War 2 - A Primer - Part 1

11 comments in this topic

Recommended Posts

397
[STORM]
Alpha Tester
1,052 posts
3,522 battles

Forward

My name is anomaly and I have a background in engineering. Recently I've been on a cryptography kick especially with regards to WW2 era cryptography. I'd like to do a series discussing the different systems these men and women used to pass messages that would define the war. These people on all sides fought to break their enemies and to defend their security. At many many points were the lives of millions hung in their hands, and the greatest turning points in the war were often preceded by a broken code. In this first article I wanted to do a bit of a primer to introduce casual readers to the world of cryptography. It starts with basic substitution methods and ends with the rotor system. The rotor being the bread and butter of all the systems I will be discussing. 

Part 2: Enigma - The German Machine

Part 3: SIGABA - The American Big Machine

Part 4: TypeX - A British Solution

Mary Queen of Scots

The basis of cryptography is to manipulate a string of letters into a different string of letters. Substituting one letter for another, for a symbol, or for a picture. Mary Queen of Scots had an entire language where her cryptographers created their own unique letters to represent english letters. Afterall, how do you break a made-up alphabet? Sir Francis Walsingham, the head of Elizabeth's secret service had the inspired idea to bring a mathematical professor into his fold. This man cracked the code by looking at many letters sent between Mary and her cohorts. When he had enough examples of hundreds if not tens of thousands of letters he went to work. He hypothesized that some letters appear more often then others, by understanding that letters like 'e' and 'a' occur far more frequently then 'w' and 'u' he gradually cracked Mary's language letter by letter.

What failed in Mary's system?

First off is arrogance, the idea that her system was unbreakable because the concept was that clever. This is important, especially in this article because Enigma, PURPLE, and JP-25 all suffered from inventors who believed them invulnerable, hundreds of thousands paid the price for their hubris.

Secondly, her cryptographic cypher, the symbols for the letters, never ever changed. If a slash meant 'a' it always meant 'a', her cohorts never changed that. Given enough methods a statistical attack is made easier. This is called cryptographic 'depth'. A good cryptographic system needs to minimize the cryptographic depth they expose themselves to.

Simple Substitution Cypher

A simple substitution cypher is where one letter is changed to another. The simplest is where you advance letters a given number of spaces. For example, if we have an advancement of three then the cypher becomes:

 

Simple Substitution Cypher

Plain Text

Cypher - Text

A

D

B

E

C

F

D

A

E

B

F

C

A message reading: FAB becomes: CDE

What is the flaw with this cypher?

It is far too easy to crack. If you crack one letter you crack all of them.

 

Lets make things a bit tougher. Lets 'randomize' (there is rarely, if ever a true random, we try to simulate it but we can only get close) the changes between letters

Plain Text

Cypher Text

A

F

B

C

C

A

D

E

E

B

F

D

FAB becomes DFC

This is better but there is a fundamental flaw. Can you guess what it is?

 

Spoiler

No letter in the cypher can be encyphered as itself. In the process of cracking the code if I have a result that has the letter D in the first position, F in the second, or C in the third then I know it HAS to be wrong. The longer a message or depth gets the more that this flaw will rear its ugly head. 

To fix that flaw lets throw something fun in the mix

Plain Text

Cypher Text

A

C

B

B

C

F

D

A

E

D

F

E

 

FAB becomes ECB

By allowing B to encrypt into itself I get rid of the fundamental flaw that none of the letters encrypt as themselves. This means that my opponent can no longer assume that any plain text solution where a letter is the same as the cypher text is a wrong solution. In the world of cyptography having the right solution is the best thing, but knowing what you have is the wrong one is just as important.

 

So we have come up with a cryptographic key that has three good things:

  1. The letter advancement is pseudo-random
  2. The letters are non reversible. A turns to C but C does not turn to A, instead it turns to F* 
  3. At least one letter encrypts as itself, in this case B encrypts to B

* This can be a flaw. A reversible system means that to decrypt your message you can put it through the exact same circuit. 

Now that you all understand some basics lets talk rotors

The fundamental flaw with this key is that the more we use it the easier it is to break. A message of 6 letters might take a little while. But for a 500 letter message or longer there is so much cryptographic depth that it would takes a professional minutes to break. And that's with a coffee break thrown in. 

How do you create a cypher key with 26 letters that is robust enough to not be broken in the first message? The answer is, YOU CANT. That's right, even by WW2 standards if you made a crypto key and typed up a report, just that one report has more then enough depth for a semi-determined enemy to easily crack. 

The solution is a key that changes continuously. Enter, the rotor machine.

The Rotor Machine

Cryptographic rotors are disc shaped devices. At this point in time most only had the letters of their alphabets, no capitals, no spaces, no numbers. Each disk had contacts on each side. The contacts would be connected by wires criss crossing. So the input contact would be connected to an output contact somewhere else on the rotor.

image.png.468a3d21b76c5a103b47d6378420e30f.png

"But anomaly, when all is said and done you've really just made an overcomplicated method to transmute a D to an E, how is this different then before?"

I am GLAD you asked. The brilliance of the rotors is that every time you press a key the signal transmits the letter, and then a ratchet system rotates the first rotor one time. 

image.png.136e3f89d4a24c5ca750ef38fe319a01.png

In the picture above the input message is: BB

The cypher text though is: HC

The key has changed! Every time you press a letter the first rotor rotates. For a given number of rotations of the first rotor the second rotor will rotate once, and so on to the third rotor. Assuming a simple setup of the second rotor rotating once for every 26 rotations of the first, and the third once every 26 of the second, you would need to type 17,576 letters (26 letter alphabet) to come back to the original position. Every letter has a unique key. Which means your cryptographic depth on any given key in a 15,000 letter document is a single letter! We've gone from a depth of hundreds if not thousands of letters with the keys we made before to a depth of ONE. Fantastic!

At this point I think you understand the basics

 

Part 2 and beyond will be located in the history section 

 

 

  • Cool 3

Share this post


Link to post
Share on other sites
1,385
Alpha Tester
5,867 posts
2,866 battles
1 hour ago, Cooper_Capt said:

USA used Navajo code talkers in WW2.   No Engineers needed.   

Not only that, but it was the only code used in the war that was never broken.

Share this post


Link to post
Share on other sites
397
[STORM]
Alpha Tester
1,052 posts
3,522 battles
9 hours ago, Cooper_Capt said:

USA used Navajo code talkers in WW2.   No Engineers needed.   

Actually thats a common misconception. The main advantages of the Navajo code talkers was the ability to relay messages as they heard them, no need to wait for someone to laboriously encrypt a message and then decrypt. They could encode and transmit in one go allowing front line officers to communicate in as real time as you can get. The other advantage is that there is no written language and it sounds like gibberish to the Japanese whom often assumed it was gibberish. 

The problem is that they suffer from the same vulnerabilities as Mary queen of Scots. It’s only a flimsy substitution cypher. On a battlefield your enemy probably won’t be able to make recordings for crypto analysts so that’s okay. But for strategic distance communications pretty much every crypto agency in use at the time would have eventually, inevitably cracked that code. Like a good Yamato player, the US recognized the fundamental flaw in their system and never put it in a position to be exploited  

The US made use of several mechanical enciphering tools. The best and most complex of which was SIGABA also known as the Converter M-134. This was the primary ciphering machine used by the US on strategic levels with ground level support being performed by more mobile cyphers including the Navajo code talkers. 

7 hours ago, 1Sherman said:

Not only that, but it was the only code used in the war that was never broken.

Not actually true. SIGABA went through the entirety of the Second World War, into the 50’s and was never cracked. In fact the US sold many of its tactical cyphering machines post war but jealously guarded SIGABA machines. Storing them in vaults under continuous guard and when in foreign countries (including allied) under guard and wired with explosives. All because it was so elaborate even American cryptographers who built it could not break a SIGABA code. The nightmare was that another country steals and copies it and will then also have impenetrable codes. 

The British TypeX, basically a further improved Enigma, also likely went through the war unbroken. It has been suggested that Germany successfully cracked a few messages during lapses in discipline but as a system very little if any actionable intelligence came from it. 

Share this post


Link to post
Share on other sites
1,385
Alpha Tester
5,867 posts
2,866 battles
1 hour ago, 1An0maly1 said:

Actually thats a common misconception. The main advantages of the Navajo code talkers was the ability to relay messages as they heard them, no need to wait for someone to laboriously encrypt a message and then decrypt. They could encode and transmit in one go allowing front line officers to communicate in as real time as you can get. The other advantage is that there is no written language and it sounds like gibberish to the Japanese whom often assumed it was gibberish. 

The problem is that they suffer from the same vulnerabilities as Mary queen of Scots. It’s only a flimsy substitution cypher. On a battlefield your enemy probably won’t be able to make recordings for crypto analysts so that’s okay. But for strategic distance communications pretty much every crypto agency in use at the time would have eventually, inevitably cracked that code. Like a good Yamato player, the US recognized the fundamental flaw in their system and never put it in a position to be exploited  

 

I read one time (admittedly on Wikipedia, but still) that apparently Navajo code talkers deployed with the Marines on Iwo Jima sent 800 messages without error once they hit the beaches, providing data that turned out to be part of the reason why the attack on the island was a success. I also read that the Navajo language is so grammatically complex that pretty much the only ways to be able to understand it are if you either grew up speaking it or if you spent a very long time learning it and were exposed to it daily.

1 hour ago, 1An0maly1 said:

Not actually true. SIGABA went through the entirety of the Second World War, into the 50’s and was never cracked. In fact the US sold many of its tactical cyphering machines post war but jealously guarded SIGABA machines. Storing them in vaults under continuous guard and when in foreign countries (including allied) under guard and wired with explosives. All because it was so elaborate even American cryptographers who built it could not break a SIGABA code. The nightmare was that another country steals and copies it and will then also have impenetrable codes. 

The British TypeX, basically a further improved Enigma, also likely went through the war unbroken. It has been suggested that Germany successfully cracked a few messages during lapses in discipline but as a system very little if any actionable intelligence came from it. 

Mind if I ask what your sources are? I haven't heard much about this and I'm curious where you're getting it.

Share this post


Link to post
Share on other sites
397
[STORM]
Alpha Tester
1,052 posts
3,522 battles
55 minutes ago, 1Sherman said:

I read one time (admittedly on Wikipedia, but still) that apparently Navajo code talkers deployed with the Marines on Iwo Jima sent 800 messages without error once they hit the beaches, providing data that turned out to be part of the reason why the attack on the island was a success. I also read that the Navajo language is so grammatically complex that pretty much the only ways to be able to understand it are if you either grew up speaking it or if you spent a very long time learning it and were exposed to it daily.

Mind if I ask what your sources are? I haven't heard much about this and I'm curious where you're getting it.

The problem isn’t with the language itself, the crypto logical alphabet, the issue is actually more to do with the fact that it never changes. For example, Navajo has no word for “tank” so they used “turtle”. If you transmitted over a strategic transmission that you were deploying a turtle division in X location, your enemy will eventually suss out what “turtle” means. In preparing for the battle of Midway the Japanese used a place holder word for Midway island. The US, not knowing what the placeholder was had Midway transmit in clear text they were low on water (or something like that). When the Japanese transmitted that *placeholder name* was low on water the US then knew for sure that they were talking about Midway. We all know what happened next. 

Granted it’s a gross oversimplification but it represents how if you don’t constantly change your cryptographic key it will inevitably be broken. On a tactical level it works well because tactical orders are executed in short time spans from minutes to a day. By the time an enemy decrypts your message it’s far too late. But for messages describing the positions of submarines for example once the code is broken you will end up with many dead submarines (a relevant example since the cracking of the three rotor enigma led to the end of the first u boat golden age)

 

Sure. The main source that sparked my interest is centered around the SIGABA. 

SIGABA: A Beautiful Idea

center for cryptological history

national security agency

During the war American cryptologists scoured German and Japanese transmissions for indications that they managed to break the code. A number of transmissions followed by interrogations of prisoners of war during and after the war including the German code breakers themselves indicated that little if any progress was made. During the closing year of the war the German code breaking group in their diary wrote that they ceased work on the American 5 letter machine as being unprofitable  implying they were focusing instead on the weaker field cyphers and typex 

its publically available via google. It’s a fascinating read. Other sources include the ubiquitous Wikipedia and a few other articles. But I highly recommend starting with the one I mentioned above. 

Share this post


Link to post
Share on other sites
1,385
Alpha Tester
5,867 posts
2,866 battles
2 hours ago, 1An0maly1 said:

The problem isn’t with the language itself, the crypto logical alphabet, the issue is actually more to do with the fact that it never changes. For example, Navajo has no word for “tank” so they used “turtle”. If you transmitted over a strategic transmission that you were deploying a turtle division in X location, your enemy will eventually suss out what “turtle” means. In preparing for the battle of Midway the Japanese used a place holder word for Midway island. The US, not knowing what the placeholder was had Midway transmit in clear text they were low on water (or something like that). When the Japanese transmitted that *placeholder name* was low on water the US then knew for sure that they were talking about Midway. We all know what happened next. 

Granted it’s a gross oversimplification but it represents how if you don’t constantly change your cryptographic key it will inevitably be broken. On a tactical level it works well because tactical orders are executed in short time spans from minutes to a day. By the time an enemy decrypts your message it’s far too late. But for messages describing the positions of submarines for example once the code is broken you will end up with many dead submarines (a relevant example since the cracking of the three rotor enigma led to the end of the first u boat golden age)

 

I get what you're saying. One thing I remember from The Imitation Game (yes, I know it's not accurate, but hear me out) is that once Alan Turing and the rest of Bletchley Park had that giant computer up and running, they realized that every single unencrypted message the Germans sent ended with the sendoff "Heil Hitler". Assuming that the coded messages they intercepted ended the same way was what allowed them to break the Enigma cypher, at least in the movie. 

The thing about the Navajo code, however, is that they weren't just placeholder names. They were placeholder names in a language that basically no one outside of the code talkers themselves can understand. Navajo code talkers would be on both ends, sending and receiving messages that they would then translate to English for their superiors. If the Japanese intercepted one of those transmissions, all they'd find is a language that they'd never be able to decipher.

2 hours ago, 1An0maly1 said:

Sure. The main source that sparked my interest is centered around the SIGABA. 

SIGABA: A Beautiful Idea

center for cryptological history

national security agency

During the war American cryptologists scoured German and Japanese transmissions for indications that they managed to break the code. A number of transmissions followed by interrogations of prisoners of war during and after the war including the German code breakers themselves indicated that little if any progress was made. During the closing year of the war the German code breaking group in their diary wrote that they ceased work on the American 5 letter machine as being unprofitable  implying they were focusing instead on the weaker field cyphers and typex 

its publically available via google. It’s a fascinating read. Other sources include the ubiquitous Wikipedia and a few other articles. But I highly recommend starting with the one I mentioned above. 

Thanks. I'll keep it in mind.

Share this post


Link to post
Share on other sites
397
[STORM]
Alpha Tester
1,052 posts
3,522 battles
45 minutes ago, 1Sherman said:

I get what you're saying. One thing I remember from The Imitation Game (yes, I know it's not accurate, but hear me out) is that once Alan Turing and the rest of Bletchley Park had that giant computer up and running, they realized that every single unencrypted message the Germans sent ended with the sendoff "Heil Hitler". Assuming that the coded messages they intercepted ended the same way was what allowed them to break the Enigma cypher, at least in the movie. 

The thing about the Navajo code, however, is that they weren't just placeholder names. They were placeholder names in a language that basically no one outside of the code talkers themselves can understand. Navajo code talkers would be on both ends, sending and receiving messages that they would then translate to English for their superiors. If the Japanese intercepted one of those transmissions, all they'd find is a language that they'd never be able to decipher.

Thanks. I'll keep it in mind.

Keep in mind though that there are demonstrated instances where messages passed in languages that are completely make believe, where even whole words are replaced with symbols have been decrypted before. 

Decryption can successfully work against strange alphabets and languages given enough cryptographic depth. 

 

If you substituted words in the English language with each other, like instead of saying “place” you said “hamburger” at seeming random, essentially creating an alternative vocabulary. Given enough depth, as in you use it often enough that I have many documents or phrases I will eventually be able to suss our that whenever you say “hamburger” you mean “place”

this strategy of attack does work on Navajo, it is exceedingly difficult, but it does work. It’s why, to the best of my knowledge (which I am not a cryptographer so grain of salt), no one used Navajo in the same instance into the mid/late 20th century. In fact they didn’t even use it on a strategic level in world war 2 because they knew it was not very secure. 

Typical language attacks, such as what Francis Walsingham used, involve searching for words that are very commonly used. The word “a” occurs far more frequently then the word “tobacco”

you basically brute force the language. 

 

One of of the things imitation game missed was that enigma was first broken in 1932 by the Polish secret service. They built several crypto bombe’s. In 1939 in secret they presented their techniques to the British and French. Spurred by and large because Enigma added the famed plug board which meant it would take more computing resources to crack then the Poles had to attack it with. Their work served a foundation for British work. 

I haven’t heard of the heil hitler thing. I can’t confirm nor deny it. It does seem odd that they would introduce such an incredibly obvious weakness into their system.

Share this post


Link to post
Share on other sites
1,385
Alpha Tester
5,867 posts
2,866 battles
3 hours ago, 1An0maly1 said:

If you substituted words in the English language with each other, like instead of saying “place” you said “hamburger” at seeming random, essentially creating an alternative vocabulary. Given enough depth, as in you use it often enough that I have many documents or phrases I will eventually be able to suss our that whenever you say “hamburger” you mean “place”

this strategy of attack does work on Navajo, it is exceedingly difficult, but it does work. It’s why, to the best of my knowledge (which I am not a cryptographer so grain of salt), no one used Navajo in the same instance into the mid/late 20th century. In fact they didn’t even use it on a strategic level in world war 2 because they knew it was not very secure. 

Typical language attacks, such as what Francis Walsingham used, involve searching for words that are very commonly used. The word “a” occurs far more frequently then the word “tobacco”

you basically brute force the language. 

I'm sure that works if you're familiar with the language or can translate it accurately, but like I said before, there wasn't a single Japanese person alive who could understand the Navajo language and there was no way any of them were going to be able to learn it. They wouldn't even get to deciphering the placeholder words because the language barrier would be impenetrable, even against the "brute force" approach you describe. How are you supposed to break a code if you can't even understand the language it's in?

3 hours ago, 1An0maly1 said:

I haven’t heard of the heil hitler thing. I can’t confirm nor deny it. It does seem odd that they would introduce such an incredibly obvious weakness into their system.

I don't know if it's true either. I just know it was an interesting part of the movie.

Share this post


Link to post
Share on other sites
Members
297 posts
2,405 battles
On 12/20/2018 at 2:20 PM, 1Sherman said:

I'm sure that works if you're familiar with the language or can translate it accurately, but like I said before, there wasn't a single Japanese person alive who could understand the Navajo language and there was no way any of them were going to be able to learn it. They wouldn't even get to deciphering the placeholder words because the language barrier would be impenetrable, even against the "brute force" approach you describe. How are you supposed to break a code if you can't even understand the language it's in?

Well, you could add archaeologists and linguists to your team. Here's an example of them doing it with limited language samples and little context - in wartime, a collection of intercepted reports coupled with what's actually happening on the battlefield will give you lots of evidence to work with.

I think the main point is that you can get around language barriers, even extremely difficult ones like those involving languages that are made-up or extinct. The weakness of the cryptographic system still shows through, which is why the US couldn't keep using this trick for long.

Edited by yungpanda

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×