Four-square Cryptanalysis
Like most pre-modern era ciphers, the four-square cipher can be easily cracked if there is enough text. Obtaining the key is relatively straightforward if both plaintext and ciphertext are known. When only the ciphertext is known, brute force cryptanalysis of the cipher involves searching through the key space for matches between the frequency of occurrence of digrams (pairs of letters) and the known frequency of occurrence of digrams in the assumed language of the original message.
Cryptanalysis of four-square generally involves pattern matching on repeated monographs. This is only the case when the two plaintext matrices are known. A four-square encipherment usually uses standard alphabets in these matrices but it is not a requirement. If this is the case, then certain words will always produce single-letter ciphertext repeats. For instance, the word MI LI TA RY will always produce the same ciphertext letter in the first and third positions regardless of the keywords used. Patterns like these can be cataloged and matched against single-letter repeats in the ciphertext. Candidate plaintext can then be inserted in an attempt to uncover the ciphertext matrices.
Unlike the Playfair cipher, a four-square cipher will not show reversed ciphertext digraphs for reversed plaintext digraphs (e.g. the digraphs AB BA would encrypt to some pattern XY YX in Playfair, but not in four-square). This, of course, is only true if the two keywords are different. Another difference between four-square and Playfair which makes four-square a stronger encryption is the fact that double letter digraphs will occur in four-square ciphertext.
By all measures, four-square is a stronger system for encrypting information than Playfair. However, it is more cumbersome because of its use of two keys and preparing the encryption/decryption sheet can be time consuming. Given that the increase in encryption strength afforded by four-square over Playfair is marginal and that both schemes are easily defeated if sufficient ciphertext is available, Playfair has become much more common.
A good tutorial on reconstructing the key for a four-square cipher can be found in chapter 7, "Solution to Polygraphic Substitution Systems," of Field Manual 34-40-2, produced by the United States Army.
Read more about this topic: Four-square Cipher