Overview
A significant part of systems security can be testing, and this presents a real challenge for most systems security engineers. Whether it is pen testing, forensic analysis, fuzz testing, or network testing, there can be infinite variations of System Under Test (SUT) when combined with the necessary testing variations.
The challenge is to develop an approach to this testing that provides the necessary flexibility without imposing an undue burden on the systems security engineer. Traditionally the options included packaged security tools; which provides an easy to use interface for pre-configured tests, and writing tests in some high level language; which provide a high degree of flexibility with a relatively high level of effort / learning curve. The downside with the packaged tools is lack of flexibility and cost.
An approach which has been increasingly more popular is to take either one (or both) of these approaches and improve through the use of Python.
For Example
A few examples of books that take this approach include:
- Grayhat Python – ISBN 978-1593271923
- Blackhat Python – ISBN 978-1593275907
- Violent Python – ISBN 978-1597499576
- Python Penetration Testing Essentials – ISBN 978-1784398583
- Python Web Penetration Testing Cookbook – ISBN 978-1784392932
- Hacking Secret Ciphers with Python – ISBN 978-1482614374
- Python Forensics – ISBN 978-0124186767
In general, these take the approach of custom code based on generic application templates, or scripted interfaces to security applications.
Conclusions
After skimming a few of these books and some of the code samples, it is become obvious that Python has an interesting set of characteristics that make it a better language for systems work (including systems security software) than any other language I am aware of.
Over the last few decades, I have learned and programmed in a number languages including Basic, Fortran 77, Forth C, Assembly, Pascal (and Delphi), and Java. Through all of these languages I have come to accept that each one of these languages had a set of strengths and issues. For example, Basic was basic. it provided a very rudimentary set of language features, and limited libraries which meant there often a very limited number of ways to do anything (and sometimes none). It was interpreted, so that meant it was slow (way back when). It was not scaleable, which encouraged small programs, and it was fairly easy to read. The net result is that Basic was used as a teaching language, suitable for small demonstration programs – and it fit that role reasonably well.
On the other hand Java (and other strongly typed language) are by nature, painful to write in due that strongly typed nature, but also make syntax errors less likely (after tracking down all of the missing semi-colons, matching braces, and type matching). Unfortunately, syntactical errors are usually the much simpler class of problems in a program.
Another attribute of Java (and other OO languages) is the object oriented capabilities – which really do provide advantages for upwards scaleability and parallel development, but result in very difficult imperative development (procedural). Yes – everything can be an object, but that does not mean that it is the most effective way to do it.
Given that background, I spent a week (about 20 hours of it) reading books and writing code in Python. In that time I went from “hello world.py” to a program with multiple classes that collected metrics for each file in a file system, placed that data in a dictionary of objects and wrote out / retrieved from a file in about a 100 lines of code. My overall assessment:
- The class / OO implementation is powerful, and sufficiently ‘weakly typed’ that it is easily useable.
- The dictionary functionality is very easy to use, performs well, massively flexible, and becomes the go-to place to put data.
- The included standard libraries are large and comprehensive, and these are dwarfed by the massive, high quality community developed libraries.
- Overall – In one week, Python has become my default language of choice for Systems Security Engineering.
Postscript
Also of note, I looked at numerous books on Python and have discovered that:
- There are a massive number of books purportedly for learning Python.
- They are also nearly universally low value, with a few exceptions.
My criteria for this low value assessment is based on the number of “me-too” chapters. For example, every book I looked at for learning Python has at least one chapter on:
- finding python and installing it
- interactive mode of the Python interpreter
- basic string functions
- advanced string functions
- type conversions
- control flow
- exceptions
- modules, functions
- lists, tuples, sets and dicts
In addition each of these sections provide a basic level of coverage, and are virtually indistinguishable from a corresponding chapter in dozens of other books. Secondary to that there was usually minimal or basic coverage of dicts, OO capabilities, and module capabilities. I wasted a lot of time looking for something that provided a more terse coverage of the basic concepts and a more complete coverage of more advanced features of Python. My recommendation to authors of computer programming books: if your unique content is much less than half of your total content, don’t publish.
From this effort I can recommend the following books:
- The Quick Python Book (ISBN 978-1935182207): If you skip the very basic parts, there is a decent level of useful Python content for the experienced programmer.
- Introducing Python (ISBN 978-1449359362): Very similar to the Quick Python book, with some unique content.
- Python Pocket Reference (ISBN 978-1449357016): Simply a must have for any language. If O’Reilly has one, you should have it.
- Learning Python (ISBN 978-1449355739): A 1500 page book that surprised me. It does have the basic “me-too” chapters, but has a number of massive sections not found in any other Python book. Specifically, Functions and Generators (200 pages), Modules in depth (120 pages), Classes and OOP in depth (300 pages), Exceptions in depth (80 pages), and 250 pages of other Advanced topics. Overall it provides the content of at least three other books on Python, in a coherent package.
Note – Although I could have provided links on Amazon for each of these books (every one of them is available at Amazon), my purpose is to provide some information on these books as resources (not promote Amazon). I buy many books directly from O’Reilly (they often have half off sales), Amazon, and Packt.