This is a post written for my Spring 2016 Number Theory class at Brown University. It was written and originally presented from a Jupyter Notebook, and changed for presentation on this site. There is a version of the notebook available at github.
Cryptography¶
Recall the basic setup of cryptography. We have two people, Anabel and Bartolo. Anabel wants to send Bartolo a secure message. What do we mean by "secure?" We mean that even though that dastardly Eve might intercept and read the transmitted message, Eve won't learn anything about the actual message Anabel wants to send to Bartolo. Before the 1970s, the only way for Anabel to send Bartolo a secure message required Anabel and Bartolo to get together beforehand and agree on a secret method of communicating. To communicate, Anabel first decides on a message. The original message is sometimes called the plaintext. She then encrypts the message, producing a ciphertext. She then sends the ciphertext. If Eve gets hold of hte ciphertext, she shouldn't be able to decode it (unless it's a poor encryption scheme). When Bartolo receives the ciphertext, he can decrypt it to recover the plaintext message, since he agreed on the encryption scheme beforehand.Caesar Shift¶
The first known instance of cryptography came from Julius Caesar. It was not a very good method. To encrypt a message, he shifted each letter over by 2, so for instance "A" becomes "C", and "B" becomes "D", and so on. Let's see what sort of message comes out.alpha = "abcdefghijklmnopqrstuvwxyz".upper()
punct = ",.?:;'\n "
from functools import partial
def shift(l, s=2):
l = l.upper()
return alpha[(alpha.index(l) + s) % 26]
def caesar_shift_encrypt(m, s=2):
m = m.upper()
c = "".join(map(partial(shift, s=s), m))
return c
def caesar_shift_decrypt(c, s=-2):
c = c.upper()
m = "".join(map(partial(shift, s=s), c))
return m
print "Original Message: HI"
print "Ciphertext:", caesar_shift_encrypt("hi")
m = """To be, or not to be, that is the question:
Whether 'tis Nobler in the mind to suffer
The Slings and Arrows of outrageous Fortune,
Or to take Arms against a Sea of troubles,
And by opposing end them."""
m = "".join([l for l in m if not l in punct])
print "Original Message:"
print m
print
print "Ciphertext:"
tobe_ciphertext = caesar_shift_encrypt(m)
print tobe_ciphertext
print "Decrypted first message:", caesar_shift_decrypt("JK")
print "Decrypted second message:"
print caesar_shift_decrypt(tobe_ciphertext)
Is this a good encryption scheme? No, not really. There are only 26 different things to try. This can be decrypted very quickly and easily, even if entirely by hand.
Substitution Cipher¶
A slightly different scheme is to choose a different letter for each letter. For instance, maybe "A" actually means "G" while "B" actually means "E". We call this a substitution cipher as each letter is substituted for another.import random
permutation = list(alpha)
random.shuffle(permutation)
# Display the new alphabet
print alpha
subs = "".join(permutation)
print subs
def subs_cipher_encrypt(m):
m = "".join([l.upper() for l in m if not l in punct])
return "".join([subs[alpha.find(l)] for l in m])
def subs_cipher_decrypt(c):
c = "".join([l.upper() for l in c if not l in punct])
return "".join([alpha[subs.find(l)] for l in c])
m1 = "this is a test"
print "Original message:", m1
c1 = subs_cipher_encrypt(m1)
print
print "Encrypted Message:", c1
print
print "Decrypted Message:", subs_cipher_decrypt(c1)
print "Original message:"
print m
print
c2 = subs_cipher_encrypt(m)
print "Encrypted Message:"
print c2
print
print "Decrypted Message:"
print subs_cipher_decrypt(c2)
Is this a good encryption scheme? Still no. In fact, these are routinely used as puzzles in newspapers or puzzle books, because given a reasonably long message, it's pretty easy to figure it out using things like the frequency of letters.
For instance, in English, the letters RSTLNEAO are very common, and much more common than other letters. So one might start to guess that the most common letter in the ciphertext is one of these. More powerfully, one might try to see which pairs of letters (called bigrams) are common, and look for those in the ciphertext, and so on.
From this sort of thought process, encryption schemes that ultimately rely on a single secret alphabet (even if it's not our typical alphabet) fall pretty quickly. So... what about polyalphabetic ciphers? For instance, what if each group of 5 letters uses a different set of alphabets?
This is a great avenue for exploration, and there are lots of such encryption schemes that we won't discuss in this class. But a class on cryptography (or a book on cryptography) would certainly go into some of these. It might also be a reasonable avenue of exploration for a final project.
The German Enigma¶
One very well-known polyalphabetic encryption scheme is the German Enigma used before and during World War II. This was by far the most complicated cryptosystem in use up to that point, and the story of how it was broken is a long and tricky one. Intial successes towards breaking the Enigma came through the work of Polish mathematicians, fearful (and rightfully so) of the Germans across the border. By 1937, they had built replicas and understood many details of the Enigma system. But in 1938, the Germans shifted to a more secure and complicated cryptosystem. Just weeks before the German invasion of Poland and the beginning of World War II, Polish mathematicians sent their work and notes to mathematicians in France and Britain, who carried out this work. The second major progress towards breaking the Enigma occurred largely in Bletchley Park in Britain, a communication center with an enormous dedicated effort to breaking the Enigma. This is where the tragic tale of Alan Turing, recently popularized through the movie The Imitation Game, begins. This is also the origin tale for modern computers, as Alan Turing developed electromechanical computers to help break the Enigma. The Enigma worked by having a series of cogs or rotors whose positions determined a substitution cipher. After each letter, the positions were changed through a mechanical process. An Enigma machine is a very impressive machine to look at [and the "computer" Alan Turing used to help break them was also very impressive]. Below, I have implemented an Enigma, by default set to 4 rotors. I don't expect one to understand the implementation. The interesting part is how meaningless the output message looks. Note that I've kept the spacing and punctuation from the original message for easier comparison. Really, you wouldn't do this. The plaintext used for demonstration is from Wikipedia's article on the Enigma.from random import shuffle,randint,choice
from copy import copy
num_alphabet = range(26)
def en_shift(l, n): # Rotate cogs and arrays
return l[n:] + l[:n]
class Cog: # Each cog has a substitution cipher
def __init__(self):
self.shuf = copy(num_alphabet)
shuffle(self.shuf) # Create the individual substition cipher
return # Really, these were not random
def subs_in(self,i): # Perform a substition
return self.shuf[i]
def subs_out(self,i): # Perform a reverse substition
return self.shuf.index(i)
def rotate(self): # Rotate the cog by 1.
self.shuf = en_shift(self.shuf, 1)
def setcog(self,a): # Set up a particular substitution
self.shuf = a
class Enigma:
def __init__(self, numcogs,readability=True):
self.readability = readability
self.numcogs = numcogs
self.cogs = []
self.oCogs = [] # "Original Cog positions"
for i in range(0,self.numcogs): # Create the cogs
self.cogs.append(Cog())
self.oCogs.append(self.cogs[i].shuf)
refabet = copy(num_alphabet)
self.reflector = copy(num_alphabet)
while len(refabet) > 0: # Pair letters in the reflector
a = choice(refabet)
refabet.remove(a)
b = choice(refabet)
refabet.remove(b)
self.reflector[a] = b
self.reflector[b] = a
def print_setup(self): # Print out substituion setup.
print "Enigma Setup:\nCogs: ",self.numcogs,"\nCog arrangement:"
for i in range(0,self.numcogs):
print self.cogs[i].shuf
print "Reflector arrangement:\n",self.reflector,"\n"
def reset(self):
for i in range(0,self.numcogs):
self.cogs[i].setcog(self.oCogs[i])
def encode(self,text):
t = 0 # Ticker counter
ciphertext=""
for l in text.lower():
num = ord(l) % 97
# Handle special characters for readability
if (num>25 or num<0):
if (self.readability):
ciphertext += l
else:
pass
else:
# Pass through cogs, reflect, then return through cogs
t += 1
for i in range(self.numcogs):
num = self.cogs[i].subs_in(num)
num = self.reflector[num]
for i in range(self.numcogs):
num = self.cogs[self.numcogs-i-1].subs_out(num)
ciphertext += "" + chr(97+num)
# Rotate cogs
for i in range(self.numcogs):
if ( t % ((i*6)+1) == 0 ):
self.cogs[i].rotate()
return ciphertext.upper()
plaintext="""When placed in an Enigma, each rotor can be set to one of 26 possible positions.
When inserted, it can be turned by hand using the grooved finger-wheel, which protrudes from
the internal Enigma cover when closed. So that the operator can know the rotor's position,
each had an alphabet tyre (or letter ring) attached to the outside of the rotor disk, with
26 characters (typically letters); one of these could be seen through the window, thus indicating
the rotational position of the rotor. In early models, the alphabet ring was fixed to the rotor
disk. A later improvement was the ability to adjust the alphabet ring relative to the rotor disk.
The position of the ring was known as the Ringstellung ("ring setting"), and was a part of the
initial setting prior to an operating session. In modern terms it was a part of the
initialization vector."""
# Remove newlines for encryption
pt = "".join([l.upper() for l in plaintext if not l == "\n"])
# pt = "".join([l.upper() for l in plaintext if not l in punct])
x=enigma(4)
#x.print_setup()
print "Original Message:"
print pt
print
ciphertext = x.encode(pt)
print "Encrypted Message"
print ciphertext
print
# Decryption and encryption are symmetric, so to decode we reset and re-encrypt.
x.reset()
decipheredtext = x.encode(ciphertext)
print "Decrypted Message:"
print decipheredtext
The advent of computers brought in a paradigm shift in approaches towards cryptography. Prior to computers, one of the ways of maintaining security was to come up with a hidden key and a hidden cryptosystem, and keep it safe merely by not letting anyone know anything about how it actually worked at all. This has the short cute name security through obscurity. As the number of type of attacks on cryptosystems are much, much higher with computers, a different model of security and safety became necessary.
It is interesting to note that it is not obvious that security through obscurity is always bad, as long as it's really really well-hidden. This is relevant to some problems concerning current events and cryptography.
Public Key Cryptography¶
The new model begins with a slightly different setup. We should think of Anabel and Bartolo as sitting on opposite sides of a classroom, trying to communicate securely even though there are lots of people in the middle of the classroom who might be listening in. In particular, Anabel has something she wants to tell Bartolo. Instead of keeping the cryptosystem secret, Bartolo tells everyone (in our metaphor, he shouts to the entire classroom) a public key K, and explains how to use it to send him a message. Anabel uses this key to encrypt her message. She then sends this message to Bartolo. If the system is well-designed, no one will be able to understand the ciphertext even though they all know how the cryptosystem works. This is why the system is called Public Key. Bartolo receives this message and (using something only he knows) he decrypts the message. We will learn one such cryptosystem here: RSA, named after Rivest, Shamir, and Addleman — the first major public key cryptosystem.RSA¶
Bartolo takes two primes such as $p = 12553$ and $q = 13007$. He notes their product $$m = pq = 163276871$$ and computes $\varphi(m)$, $$\varphi(m) = (p-1)(q-1) = 163251312.$$ Finally, he chooses some integer $k$ relatively prime to $\varphi(m)$, like say $$k = 79921.$$ Then the public key he distributes is $$ (m, k) = (163276871, 79921).$$ In order to send Bartolo a message using this key, Anabel must convert her message to numbers. She might use the identification A = 11, B = 12, C = 13, ... and concatenate her numbers. To send the word CAB, for instance, she would send 131112. Let's say that Anabel wants to send the message NUMBER THEORY IS THE QUEEN OF THE SCIENCES Then she needs to convert this to numbers.conversion_dict = dict()
alpha = "abcdefghijklmnopqrstuvwxyz".upper()
curnum = 11
for l in alpha:
conversion_dict[l] = curnum
curnum += 1
print "Original Message:"
msg = "NUMBERTHEORYISTHEQUEENOFTHESCIENCES"
print msg
print
def letters_to_numbers(m):
return "".join([str(conversion_dict[l]) for l in m.upper()])
print "Numerical Message:"
msg_num = letters_to_numbers(msg)
print msg_num
So Anabel's message is the number
$$2431231215283018152528351929301815273115152425163018152913191524131529$$
which she wants to encrypt and send to Bartolo. To make this manageable, she cuts the message into 8-digit numbers,
$$24312312, 15283018, 15252835, 19293018, 15273115, 15242516, 30181529, 13191524, 131529.$$
To send her message, she takes one of the 8-digit blocks and raises it to the power of $k$ modulo $m$. That is, to transmit the first block, she computes
$$ 24312312^{79921} \equiv 13851252 \pmod{163276871}.$$
# Secret information
p = 12553
q = 13007
phi = (p-1)*(q-1) # varphi(pq)
# Public information
m = p*q # 163276871
k = 79921
print pow(24312312, k, m)
She sends this number
$$13851252$$
to Bartolo (maybe by shouting. Even though everyone can hear, they cannot decrypt it). How does Bartolo decrypt this message?
He computes $\varphi(m) = (p-1)(q-1)$ (which he can do because he knows $p$ and $q$ separately), and then finds a solution $u$ to
$$ uk = 1 + \varphi(m) v.$$
This can be done quickly through the Euclidean Algorithm.
def extended_euclidean(a,b):
if b == 0:
return (1,0,a)
else :
x, y, gcd = extended_euclidean(b, a % b) # Aside: Python has no tail recursion
return y, x - y * (a // b),gcd # But it does have meaningful stack traces
# This version comes from Exercise 6.3 in the book, but without recursion
def extended_euclidean2(a,b):
x = 1
g = a
v = 0
w = b
while w != 0:
q = g // w
t = g - q*w
s = x - q*v
x,g = v,w
v,w = s,t
y = (g - a*x) / b
return (x,y,g)
def modular_inverse(a,m) :
x,y,gcd = extended_euclidean(a,m)
if gcd == 1 :
return x % m
else :
return None
print "k, p, q:", k, p, q
print
u = modular_inverse(k,(p-1)*(q-1))
print u
In particular, Bartolo computes that his $u = 145604785$. To recover the message, he takes the number $13851252$ sent to him by Anabel and raises it to the $u$ power. He computes
$$ 13851252^{u} \equiv 24312312 \pmod{pq},$$
which we can see must be true as
$$ 13851252^{u} \equiv (24312312^{k})^u \equiv 24312312^{1 + \varphi(pq)v} \equiv 24312312 \pmod{pq}.$$
In this last step, we have used Euler's Theorem to see that
$$ 24312312^{\varphi(pq)v} \equiv 1 \pmod{pq}.$$
# Checking this power explicitly.
print pow(13851252, 145604785, m)
Now Bartolo needs to perform this process for each 8-digit chunk that Anabel sent over. Note that the work is very easy, as he computes the integer $u$ only once. Each other time, he simply computes $c^u \pmod m$ for each ciphertext $c$, and this is very fast with repeated-squaring.
We do this below, in an automated fashion, step by step.
First, we split the message into 8-digit chunks.
# Break into chunks of 8 digits in length.
def chunk8(message_number):
cp = str(message_number)
ret_list = []
while len(cp) > 7:
ret_list.append(cp[:8])
cp = cp[8:]
if cp:
ret_list.append(cp)
return ret_list
msg_list = chunk8(msg_num)
print msg_list
This is a numeric representation of the message Anabel wants to send Bartolo. So she encrypts each chunk. This is done below
# Compute ciphertexts separately on each 8-digit chunk.
def encrypt_chunks(chunked_list):
ret_list = []
for chunk in chunked_list:
#print chunk
#print int(chunk)
ret_list.append(pow(int(chunk), k, m))
return ret_list
cipher_list = encrypt_chunks(msg_list)
print cipher_list
This is the encrypted message. Having computed this, Anabel sends this message to Bartolo.
To decrypt the message, Bartolo uses his knowledge of $u$, which comes from his ability to compute $\varphi(pq)$, and decrypts each part of the message. This looks like below.
# Decipher the ciphertexts all in the same way
def decrypt_chunks(chunked_list):
ret_list = []
for chunk in chunked_list:
ret_list.append(pow(int(chunk), u, m))
return ret_list
decipher_list = decrypt_chunks(cipher_list)
print decipher_list
Finally, Bartolo concatenates these numbers together and translates them back into letters. Will he get the right message back?
alpha = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
# Collect deciphered texts into a single list, and translate back into letters.
def chunks_to_letters(chunked_list):
s = "".join([str(chunk) for chunk in chunked_list])
ret_str = ""
while s:
ret_str += alpha[int(s[:2])-11].upper()
s = s[2:]
return ret_str
print chunks_to_letters(decipher_list)
Yes! Bartolo successfully decrypts the message and sees that Anabel thinks that Number Theory is the Queen of the Sciences. This is a quote from Gauss, the famous mathematician who has been popping up again and again in this course.
Why is this Secure?¶
Let's pause to think about why this is secure. What if someone catches the message in transit? Suppose Eve is eavesdropping and hears Anabel's first chunk, $13851252$. How would she decrypt it? Eve knows that she wants to solve $$ x^k \equiv 13851252 \pmod {pq}$$ or rather $$ x^{79921} \equiv 13851252 \pmod {163276871}.$$ How can she do that? We can do that because we know to raise this to a particular power depending on $\varphi(163276871)$. But Eve doesn't know what $\varphi(163276871)$ is since she can't factor $163276871$. In fact, knowing $\varphi(163276871)$ is exactly as hard as factoring $163276871$. But if Eve could somehow find $79921$st roots modulo $163276871$, or if Eve could factor $163276871$, then she would be able to decrypt the message. These are both really hard problems! And it's these problems that give the method its security. More generally, one might use primes $p$ and $q$ that are each about $200$ digits long, and a fairly large $k$. Then the resulting $m$ would be about $400$ digits, which is far larger than we know how to factor effectively. The reason for choosing a somewhat large $k$ is for security reasons that are beyond the scope of this segment. The relevant idea here is that since this is a publicly known encryption scheme, many people have strenghtened it over the years by making it more secure against every clever attack known. This is another, sometimes overlooked benefit of public-key cryptography: since the methodology isn't secret, everyone can contribute to its security — indeed, it is in the interest of anyone desiring such security to contribute. This is sort of the exact opposite of Security through Obscurity. The nature of code being open, public, private, or secret is also very topical in current events. Recently, Volkswagen cheated in its car emissions-software and had it report false outputs, leading to a large scandal. Their software is proprietary and secret, and the deliberate bug went unnoticed for years. Some argue that this means that more software, and especially software that either serves the public or that is under strong jurisdiction, should be publicly viewable for analysis. Another relevant current case is that the code for most voting machines in the United States is proprietary and secret. Hopefully they aren't cheating! On the other side, many say that it is necessary for companies to be able to have secret software for at least some time period to recuperate the expenses that go into developing the software. This is similar to how drug companies have patents on new drugs for a number of years. This way, a new successful drug pays for its development since the company can charge above the otherwise-market-rate. Further, many say that opening some code would open it up to attacks from malicious users who otherwise wouldn't be able to see the code. Of course, this sounds a lot like trying for security through obscurity. This is a very relevant and big topic, and the shape it takes over the next few years may very well have long-lasting impacts.Condensed Version¶
Now that we've gone through it all once, we have a condensed RSA system set up with our $p, q,$ and $k$ from above. To show that this can be done quickly with a computer, let's do another right now. Let us encrypt, transmit, and decrypt the message "I have never done anything useful. No discovery of mine has made, or is likely to make, directly or indirectly, for good or ill, the least difference to the amenity of the world". First, we prepare the message for conversion to numbers.message = """I have never done anything useful. No discovery of mine has made,
or is likely to make, directly or indirectly, for good or ill, the least
difference to the amenity of the world"""
message = "".join([l.upper() for l in message if not l in "\n .,"])
print "Our message:\n"+message
Now we convert the message to numbers, and transform those numbers into 8-digit chunks.
numerical_message = letters_to_numbers(message)
print "Our message, converted to numbers:"
print numerical_message
print
plaintext_chunks = chunk8(numerical_message)
print "Separated into 8-digit chunks:"
print plaintext_chunks
We encrypt each chunk by computing $P^k \bmod {m}$ for each plaintext chunk $P$.
ciphertext_chunks = encrypt_chunks(plaintext_chunks)
print ciphertext_chunks
This is the message that Anabel can sent Bartolo.
To decrypt it, he computes $c^u \bmod m$ for each ciphertext chunk $c$.
deciphered_chunks = decrypt_chunks(ciphertext_chunks)
print "Deciphered chunks:"
print deciphered_chunks
Finally, he translates the chunks back into letters.
decoded_message = chunks_to_letters(deciphered_chunks)
print "Decoded Message:"
print decoded_message
Even with large numbers, RSA is pretty fast. But one of the key things that one can do with RSA is securely transmit secret keys for other types of faster-encryption that don't work in a public-key sense. There is a lot of material in this subject, and it's very important.
The study of sending and receiving secret messages is called Cryptography. There are lots of interesting related topics, some of which we'll touch on in class.
The study of analyzing and breaking cryptosystems is called Cryptanalysis, and is something I find quite fun. But it's also quite intense.
I should mention that in practice, RSA is performed a little bit differently to make it more secure.
Aside: this is the first time I've converted a Jupyter Notebook, and it turns out it's very easy. However, there are a few formatting annoyances that I didn't consider, but which are too minor to spend time fixing. But now I know!
Leave a comment
Info on how to comment
To make a comment, please send an email using the button below. Your email address won't be shared (unless you include it in the body of your comment). If you don't want your real name to be used next to your comment, please specify the name you would like to use. If you want your name to link to a particular url, include that as well.
bold, italics, and plain text are allowed in
comments. A reasonable subset of markdown is supported, including lists,
links, and fenced code blocks. In addition, math can be formatted using
$(inline math)$ or $$(your display equation)$$.
Please use plaintext email when commenting. See Plaintext Email and Comments on this site for more. Note also that comments are expected to be open, considerate, and respectful.
Comments (3)
2016-03-28 Just Asking
Excuse me, Professor?
In class, you said some other ways that aren't RSA, like with lots of acronyms. What were those, and how do they work with RSA?
2016-03-30 davidlowryduda
Hello!
Yes, we mentioned a few of the other big tools in cryptography right now. I think the ones with biggest interest are AES (the advanced encryption standard), its predecessor DES, and the SHA-256 hashing function. Are these what you were talking about?
It's a bit involved to really talk about them, but your favorite search engine will get you very far. I will say that AES is used frequently, but it requires both sides to agree and know a key beforehand. What this means is you use something (maybe even RSA, or something like it) to perform a key exchange, and then you might use AES from there.
It's a big, nice topic! It's definitely big and nice enough for a good final project, too.
2018-01-16 Tim H
Awesome!!