Exploit Development Framework Design

Thu, 16 Apr 2009 09:40:15 GMT
by pdp

Perl, Ruby Python: use the language that suits your character. However, one of the things that differentiate python from the rest is its philosophy, which is: "there should be one-- and preferably only one --obvious way to do it" (where "it" is "a problem"). This philosophy gives python some interesting advantages over other similar languages. That will be explained later on.

This post is merely a summary of my research work on how to build a better exploitation framework ala metasploit-style.

The Problem

Metasploit is great but there are three things that makes the framework sometimes inconvenient: it's size, it's dependency of the ruby platform and of course it's speed. It will be great if for example we can take a single exploit (or a set of exploits) out of the framework and compile it into a standalone executable. On the advantage side, this type of solution will also allow us to ship the framework as a payload to already compromised systems and use it from there as a stepping stone for further propagation. It will also allows us to run exploits from compromised embedded devices as long as we can compile for their architecture, which is pretty cool.

I understand that it is possible to bundle the entire framework plus the ruby environment into one executable but such a solution is simply not elegant enough and not fully cross-platformed.

Introduction

Back in the days when Metasploit was written in Perl, there were a few other frameworks trying to do similar things but in C and C++. A solution based on C or C++ is a lot more interesting as it allows us to compile standalone versions of the framework and use them as we wish. It simply makes the framework very good for embedding and also quite suitable for delivering it as a payload to the systems we would like to compromise.

Nowadays, a C and C++ solution is often doomed to failure. The reason for this is because when building a framework you can easily get into a situation where you need to solve a pretty complicated problem. Both C and C++ lack the dynamicism and the degree of expression available in languages such as perl, python and ruby and therefore, while they remain very suitable for low level stuff, they start to loose their grounds when it is needed to build something that is more abstract and high level.

Some Solutions

Keeping all of the above in mind I started putting words into practice. In the spirit of a zen monk, I started thinking which parts of the metasploit framework are most valuable to a penetration tester so that they can be branched out. As it happens, the obvious answer is: "the exploits". The "Auxiliary" modules are great but they represent functionalities which are already available in other tools. So, the first idea was to take the exploits and payloads out and rewrite them into something that is more suitable.

I decided to see for myself if I can prototype a simple exploitation framework in C++ that all it should so is to implement several abstract interfaces for exploit development, a a class with common methods for payloads (empty of course) and of course a simple interface to run an exploit with a payload against a target. All of this was achieved in a "hello world" fashion exploiting a simple stack overflow on a proggie from the command line and of course without the need to circumvent any protection mechanisms in place.

Although I was pleased with the result of the prototype, I was not convinced that this is a good enough solution. Programming in C++ is fun, especially when you haven't done it for a couple of years, but still not as practicle as I would like it to be. We can most certainly build a DSL on the top of C and C++ by using Preprocessor Directives but when you are developing an exploit you want to make the process as painless as possible and C directives are only making it worse when hunting for a bug in the exploit. Not to mention that compiling something every time you make a change is not cool at all.

Being a pythonist and knowing the python mantra inside out, I thought that it should be possible to write all of the exploits and payloads in python and convert them into C or C++ at later stage as long as I stick to using a minimal set of the language features which can be directly translated with regexes and some basic parsing. After all, python looks like an executable pseudo code. Luckily for me, such a solution already exists and it is called shedskin.

Now shedskin is a lot more than a simple python to C++ translator. Not only it can convert a python program to C++ source but it also implements all of python's builtins and it has support for some of python's most useful modules such as re and socket. On the top of that, it is trivial to implement additional modules to the shedskin framework in python. This is a product I will happily pay for!

Analysis

I played quite a lot with the shedskin compiler tweaking things as I go. Although the parser is pretty advanced there are some restrictions enforced on the language. All of them are nicely covered in the shedskin's tutorial.

It was time to see if I do need the advanced python features for developing the exploits. I run through all Metasploit payloads and exploits and a pattern started to emerge. The majority of the exploits were pretty basic. They all came down to the following algorithm more or less:

  1. Select an exploit
  2. Pack a structure/payload that will be sent over a socket or will be dumped into a file
  3. Send/Save the payload

Obviously, there is no need for python sugar to implement that.

The Design

I did quite a lot of work investigating the best approach to tackle the problem of creating a good enough exploitation framework and I came up with the following basic idea:

We start with the same basic building blocks as found in metasploit. We need abstract classes for Exploits and Shellcodes and also classes that implement them to define more functional classes such as those that needs to be implemented when writing remote exploits for example (socket stuff). We use the basic python capabilities keeping shedskin in mind. As I mentioned, shedskin is quite advanced so most of the functionalities can be implemented without even taking it into consideration at all.

That will provide the core of the framework. All of the exploits now can be written on the top of this. The exploits themselves should reuse as much as of the builtin methods as possible as that their portability will be guaranteed.

A layer above that, we write as much as python sugar as we want. We simply don't care how we are going to write it because that part of the framework doesn't have to be compiled.

In summary, we layer the whole thing like this:

  • Layer 01. Core Exploit Development Classes implementing the most basic set of python features
  • Layer 02. Exploits implementing the Core Exploit Development Classes
  • Layer 03. Python sugar to glue it all together

Conclusion

So, it is possible to write a good and well-designed exploitation framework in python that allows exploits to be separated and compiled in standalone native executables. Not only that, but we do not sacrifice from the dynamicism of the python language as while the core will be written in basic python, the rest will be as dynamic as we want. Imho, this is all possible due to python's mantra that "there should be one-- and preferably only one --obvious way to do it". It just makes it easier to write briliant tools such as shedskin.

Although I am quite excited to start writing such a beast right a way, I am going to pass this time. I am starting to learn to say "NO" because I've got far too many things on my plate already. However, if anyone is interested in working on this, I will be very happy to facilitate the project as much as I can and give a hand where necessary.

I am very interested to hear your opinion and I am even more interested to get the opinion of the Metasploit team as they have a lot more experience in coding exploitation frameworks than me.

Archived Comments

HDHD
Look forward to seeing your work -- keep in mind that many of the things that make metasploit large, unwieldy, and slow - are also what makes it so flexible and dynamic.
Kevin FitzgeraldKevin Fitzgerald
Interesting approach, python -> C++ to solve this problem is a new idea with a lot of new challenges anyway, what about running on remote systems, cross compiling?, and library version issues? A Python or Ruby virtual machine is a good solution as many people have pointed out before. Do you know Brown/Dunlop Mosquito framework? A cool screencast was released recently about injecting a Java VM as exploit payload: http://blog.netifera.com/video-the-java-virtual-machine-as-shellcode/
pdppdp
HD, I may do more experimentation with this concept but I doubt that I will convert all of these ideas into a product, simply because majority of the work I do on a daily basis is not related to this field and that plays a big factor. However, I will be happy if someone takes on the idea and develops it into something useful. Perhaps future generations of the Metasploit framework will have the core modules written in a subset of ruby to make them easy for exporting and compiling into self-sustaining modules? It is possible! Kevin, virtual machines are excellent. I am currently toying with netifera and so far looks interesting and quite promising. I think that the challenge is not how to do all the things that metasploit and other frameworks do, but how to do only specific tasks in the best possible way. The key is to know what to eliminate and what to keep from the design. While I was playing around with the concepts presented in the article above I thought that it might be cool to have some kind of swissarmyknife tool which all it does is to generate shellcodes and have the capability to encode them, and also interact with them. This type of tool wont solve all problems but it will prove to be invaluable in many situations. For sure, it will allow exploits to be easily written in various languages and platforms by just wrapping around the tool. This is a small solution with potentially gigantic technological impact. Design is important! Well, this is my humble opinion only.
mattmatt
Have you tried PyPy? It also has the benefit of being free. I haven't tried using it to convert Python to C, but it's certainly one of its supported features. Let me know if you have a chance to play with it, I'd be curious to hear your experience with it.
pdppdp
I quickly played with it this morning but I couldn't make it work. I will try it over the weekend.
CarlOxCarlOx
Very Interesting post, right now i've been researching to build a new exploitation framework, is a way to contact with you? and facilitate me some information of this project? thanks.
sriramsriram
Nice analysis done! I am trying to follow your foot steps and trying to make a pythonic application (xploit playground). As you pointed out the exploits of metasploit are its heartbeat, i would also like to add that the shellcodes/shellcodegen etc are the holy grail of hacking and it needs the most attention. As exploits come and go(when patched) the only precious thing left out is the shellcode, as it can be used again and again. So in my view logically for an effective pythonic framework the first thing is to implement those payload stuff. There is a proverb 'Don't give a fish to an hungry man, just teach him fishing...' I find at least 93 payload types ready-made into the metasploit framework. Most of the exploits are simple class extension that provides metainfo and data to the payloads/payloadgens. Also the 3 layers can be made as pluggable as possible. Exploits can be made run with/without the framework or standalone exploits are very easy in a pythonic framework. Using pypy/shedskin to generate c/c++ code is awesome. This can be used to make highlevel stuff by inherent python high level features. Also modules like shellforge/impacket etc are very useful for the low level stuff. It would be even awesome if python had a disable dynamic prg switch, we can then implement stuff that pypy does. I dont know whether I can code all those stuff..but it just came to my mind.. after all i am just a guy who want to learn python and security..!
pdppdp
sriram, I think we are onto something here :) good ideas...
AndrewAndrew
hey I wanted to play around with this and used a test exploit. However the library "struct" is unavailable for the shedskin package how did you get around this. If I can figure a way around this I may pursue this project. Cheers