Skip navigation

28 July 2009

Life is an unbroken flow of emotions.  Even when we believe we are feeling nothing, we are simply detached from ourselves.  It is then important to recognize these moments and delve inward to discover and understand the person within.

The secret behind the fantasy of TV, Movie, and Video Games is to provide the people with believable life, no matter how improbable or extraordinary.  This is the reason WoW and FFXI continue to draw large crowds.  It provides people an escape to a world of complete freedom and control, despite the notable lack of these 2 key components in their real life.

A world of fantasy will always be more enticing than selling burgers at a McDonald’s.

The observation of life provides external stimulus for us to change ourselves.

When it comes to the imagination, nothing you imagine is ever wrong.

Get into the head of someone who doesn’t need control.

Try everything once.

 

Now, this is an interesting entry.  At first, it says a whole lot of nothing.  As an acting note, knowing life is an unbroken flow of emotions is important, because at no point on stage should you ever not feel something.  Then I tried to be profound and explain how feeling nothing is just some strange existential experience.  Huh.

The interesting thing here is the very accurate description of what makes media so enticing.  I would have known though, because at this point I had been playing Final Fantasy XI for nearly 7-8 years, and had spent a majority of my life parked in front of a computer/tv consuming massive amounts of media.  I believe I unhooked my TV from cable shortly after this.  I haven’t hooked it back up since.

Cutting the cord is highly recommended. 

Mambooko

Mambooko:  The native guide for adventurers, indigenous to the Amazon rain forest, his tribe protects the land.  He is a tall skinny white man with glasses.

 

Why am I here?

The animal has ravaged my lands?

Took my brother?

At first, it was my tribe that slowly killed the cattle.  The Brazilians were killing the land, and we took back what we could, but we full well knew it wouldn’t stop them.

The shaman went up the mountain, to the highest ridges, seeking for an answer.  He had been gone for 2 years, and we had lost hope.

We had all but accepted our fate when he walked out from the wilderness carrying a child.  He found the answer to our problem.

He never said where she came from, but we knew this was no ordinary child.  We had to feed her whole sides of beef as she grew, until we could teach her to hunt.  That was our mistake.

We did not know the power granted to this child, but we trained her to hunt like the panther, the tiger, the snake… then she killed a Brazilian rancher.  She devoured him down to the bones.

When we awoke that morning to a pungent smell of death, and found her lying asleep in a pool of blood, sucking on the thum of a meatless hand, we knew it was too late.

We tried to restrain her, but she killed four men before we were able to bind her legs.  As my brother tied her hands, she managed to lunge at this neck.  He died several hours later.

We sealed her in a cave with two sides of beef.  When we returned to feed her the next day, she had escaped.

I discovered her lair one day while hunting wild bore.  My tribe dared not go back to finish the job.

Every step I take back toward the monsters hiding place is a fearful one.

10 JUNE 2009

Today begins the day where I start life over.  I have not yet accepted that this pain will be forever, but I know I must relearn how to live.

I must learn to transition from a life with pain, to a life (just with pain).

In this case, the word just is ok, because the objective is to minimize the impact of the pain, not to minimize the pain completely.

I believe this is my test.  A life-long test.  A test that reminds me that I do not control everything that can happen to me, but that struggle in-and-of itself is the reward.

 

This spoke of my scoliosis, at the point at which the pain I was enduring had me limping.  I was eating Ibuprofen at more than 2400mg a day (for over a year) just to function.  Today I take no pills.  Today I am in no noticeable pain (most of the time).  It was shortly after this that I stopped taking Ibuprofen (cold turkey), and my therapies finally started to provide lasting relief.

Enter.

Today I quit Magic: The Gathering.  Well competitive MTG.  I like playing Commander from time to time, so I’m keeping ONE deck.

Everything in the deck (aside from format staples) either causes an item to enter the battlefield, has an effect upon entering the battlefield, causes a flicker effect or has an immediately dramatic effect upon entering play. The entire premise of this deck started because “Venser, The Soujorner” is my favorite card, and I love the entire flicker mechanic.

I’ve been tweaking it for the better part of 2 years, and i’m only missing about 4-5 cards (3 lands, 1 artifact, and 1 creature). Sadly they’ll double the price of the deck. Maybe someday :].

Enter

Commander
Grand Arbiter Augustin IV

Creatures
Angel of Serenity
Archaeomancer
Avacyn, Angel of Hope
Chancellor of the Spires
Deadeye Navigator
Diluvian Primordial
Djinn of Wishes
Draining Whelk
Isperia, Supreme Judge
Frost Titan
Gilded Drake
Jace’s Mindseeker
Jin-Gitaxias, Core Augur
Lighthouse Chronologist
Luminate Primordial
Mulldrifter
Phantasmal Image
Phyrexian Ingester
Phyrexian Metamorph
Solemn Simulacrum
Sphinx of Uthuun
Stonehorn Dignitary
Sun Titan
Teferi, Mage of Zhalfir
Ulamog, the Infinite Gyre
Venser, Shaper Savant

Instants
Aethermage’s Touch
Blue Sun’s Zenith
Counterspell
Cyclonic Rift
Desertion
Dissipate
Enlightened Tutor
Fact or Fiction
Ghostly Flicker
Mysical Tutor
Negate
Path to Exile

Sorcery
Bribery
Day of Judgment
Rite of Replication
Supreme Verdict
Time Warp

Artifacts
Azorius Signet
Darksteel Ingot
Gilded Lotus
Howling Mine
Lightning Greaves
Mimic Vat
Oblivion Stone
Sensei’s Divining Top
Sol Ring
Detention Sphere
Land Tax
Leyline of Anticipation
Oblivion Ring
Mind Unbound
Parallax Wave
Righteous Authority
Sunken Hope

Planeswalkers
Jace, Architect of Thought
Jace Beleren
Jace, The Mind Sculptor
Venser, the Sojourner

Lands
Azorius Chancery
Celestial Colonnade
Command Tower
Eiganjo Castle
Flood Plain
Glacial Fortress
Hallowed Fountain
Mystic Gate
Nimbus Maze
Reliquary Tower
Seachrome Coast
Sejiri Refuge
Skycloud Expanse
Strip Mine
Temple of the False God
Thespian’s Stage
14 Islands
5 Plains

Bonus Fun
Magical Hacker (Brain)
Ladies’ Knight (Brawn)
Fascist Art Director (Pretension)

Brain outsmarts Brawn
Brawn beats up Pretension
Pretension annoys Brain

It’s my version of Rock Lobster, Paper Tiger, Scissor Lizards :].

Imagine

When it comes to imagination, nothing is ever wrong.

15 MARCH 2009

Never fall into the trap of thinking you know better, and judging critically.

Elitism is a bath to burnt bridges.

Also, never get a tattoo.

The hardest thing in the world is to acknowledge that the only person who can push us to become our greatest is ourselves.  Other people may help us find the motivation along the way, but it is important to remember who makes the decision to take the first step, and who makes the decision to follow through.  When you can acknowledge that you are the person responsible for your own fate, the help you receive becomes an important part of survival.

You realize that while we are molded by the people we surround ourselves with, we are the ones that made the decision to listen to them.

 

Today I have 3 tattoos, and think this is a hilarious entry.  I also think 20 year old me was more enlightened than the now 25 year old me.

I’ll make my own PCAP Parsing API… WITH BEER… AND HOOKERS!

So I’ve spent about 8 hours or so whipping this bad boy up.  It’s actually fairly functional, but it only parses through the PCAP header and PACKET header at this point (as in the PCAP packet header, not the IP packet header)

Link to files (comes with test script and pcap): pcap_parser_0.01b.zip

As I mentioned in my last programming post, I’m really fed up with how inflexible the pure-python pcap libraries are when it comes to deciding exactly what part of a packet gets parsed.  So…

 

#! /usr/local/bin/python3
import pcap
p = pcap.Pcap_file("test.pcap")
pkt = p.next_packet()
while pkt:
    print(pkt)
    pkt = p.next_packet()

 

This bad boy does alot. With the default settings, we get…

 

PACKET( ts_sec=1374367283, ts_usec=, incl_len=66, orig_len=, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=, incl_len=125, orig_len=, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=, incl_len=66, orig_len=, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=, incl_len=491, orig_len=, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=, incl_len=66, orig_len=, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=, incl_len=342, orig_len=, endian=<,)

 

but if we add a few lines…

 

#! /usr/local/bin/python3
import pcap
import packet

pConfig = packet.PARSE_CONFIG(ts_sec=True, 
				ts_usec=True, 
				incl_len=True,
				orig_len=True)
upConfig = packet.UNPACK_CONFIG(ts_sec=True, 
				ts_usec=True, 
				incl_len=True,
				orig_len=True)

p = pcap.Pcap_file("test.pcap")
pkt = p.next_pack(pConfig=pConfig, upConfig=upConfig)
while pkt:
    print(pkt)
    pkt = p.next_pack(pConfig=pConfig, upConfig=upConfig)

 

we get…

PACKET( ts_sec=1374367283, ts_usec=850337, incl_len=125, orig_len=125, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=850478, incl_len=66, orig_len=66, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=850810, incl_len=491, orig_len=491, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=850857, incl_len=66, orig_len=66, endian=<,)
PACKET( ts_sec=1374367283, ts_usec=960149, incl_len=342, orig_len=342, endian=<,)

 

That doesn’t look like a huge difference, but the magic is really going on behind the scenes.  The difference between this and something like Scappy and DPKT is that if you don’t set values to True in the config classes, it doesn’t even read those bytes. It just moves the fuck on.

If you set them to parse but not to unpack, then they’ll stay as binary. You don’t always need to unpack, so it’s a waste of resources.

I’ll probably switch from a config class to some kind of Bitwise operations to handle configurations. So you’ll pass configuration parameters like…

pktCfg = TS_SEC | INCL_LEN | TS_SEC_UPK | INCL_LEN_UPK # 1 | 4 | 32 | 128
pkt = p.next_packet(cfg=pktCfg)

Each parser would need a default parse value, which would likely be a minimalistic approach at reaching the next header (for example, IP header would only unpack header length and next protocol value by default). Furthermore this lets a developer have control of what is parsed with flexibility, as he can simply OR/AND/XOR his default config with a new value on the fly prior to calling the parsing function.

For now…

its-happening-ron-paul-gif

Why is efficient packet inspection in Python so painful?

This is something I’ve been wondering about for a long while.

Why don’t any Pure-Python Pcap parsing APIs have the functionality I want?

I’ve been involved in building tools that parse PCAP looking for “bad things” for about 6 years, and something I’ve noticed regarding every available packet parsing library (Scappy, Dpkt, pcaputils, blah blah blah) is that they all have certain problems in common:

  • Inflexibility:

    • I don’t get to decide what parts of a header gets parsed, it just farts it all out, sometimes even unpacked.
    • Sometimes, when you go to print the data, it just decides to unpack that shit for you, but really it’s still binary when calling the actual primitives.  What kind of shit is that?!

 

  • Speed & Scalability:

    • This is actually symptom of inflexibility.
    • If you read every single byte in a header, and unpack every single thing you read, you’re gonna have a bad slow time.

 

  • Endianness Failures:

    • This is primarily an issue I have with DPKT and PCAP creation tools that don’t use standard network byte order.
    • In the case where PCAPs are stored in little endian, DPKT shits a brick when handling certain headers.  Furthermore, attempting to swap the default endianness can even worse brick shitting.
    • Endianness flexibility may sound pointless, but tell me that when you start integrating into mixed-endian environments.You will curse the existence of little endian PCAP.

 

  • Filter Support

    • Why aren’t there built in filtering functions that can be applied in such a way that I can auto-ignore certain IP addresses or IP ranges?  This supports speed and flexibility and scalability and… omg just why.  WHY.
    • To those making the argument “well, that’s parse of the thing you need to program into your product”:  These are extremely common features in any parsing program, seems like a good reason to  add it to your API.

 

  • Stream/Flow Carving

    • When it comes to security, more often than not, i’m going to care more about the content of HTTP or DNS than I am going to care about Layers 2-4.
    • I definitely care about what protocols are going over the wire, but as adversaries become more complex (and as for-sale toolkits become more sophisticated) there are simply going to be less indicators in the raw flow data.
    • So, why can’t I combine an individual session, give it a unique ID, and save that ID and flow data for later use?  There are tools that do recombine sessions, which you can then pass upper-layer data over to something like DPKTs HTTP parsers… but maybe i don’t want to do that right this second, maybe i want to move on to the next packet and not hold up everything so i can decide whether or not that HTTP session is interesting.
    • Maybe I want to put that off until later, which brings us to…

 

  • File Carving/Session Recombination

    • I have 2 PCAP files, they are 2GB each (a manageable filesize on x86).  They contain packet data from one consecutive capture event.  This means there are sessions that will start in File A that END in File B.
    • If i have pre-parsed these PCAPs and have the header data for each packet/flow, and I save the offsets for each packet, I should have an easy way to re-access those sessions later without having to keep the packet data in memory (or worse, re-parse the entire PCAP).
    • No one has built a readily available carving function that allows us zip through a PCAP file efficiently and get 1 session based on packet offsets.  This is easily the most important function of multi-tiered packet inspection on scaled systems.  Parse OSI Layers 2-4, save session identifiers, and move the fuck on.  Do deep-dive in a separate process that targets packets intelligently.
    • The worst part about this is that it takes like 10 damn lines of code to write the carving function, it’s just a matter of aggregating the offsets.

 

I actually know why these things haven’t been addressed, and it’s commendable to be honest.  The available libraries do their best to make the functionality they provide as easy to use as possible while also trying to minimize the amount of knowledge the developer needs to create something useable.

With the exception of scalability/speed, I know my issues are pretty specific.  They really only apply to someone who wants to write a packet inspection program that will scale well while still enabling deep inspection of sessions, and enabling end users (analysts) to pull back intelligently targeted sessions with a meta-data search application.

I mean… that’s pretty niche…. i think…

You know what, I’ll make my own parsing library!  With beer! And Hookers!

End. Rant.

I’ll update again when I can at least go from packet to packet and record offsets.  I have to write some basic iptools for packing/unpacking and the packet header parser.  After that I can start plagiarizing other libraries for good ethernet/ipv4/tcp/udp parsing.

 

No’Mo-Shave Log: Day 17

Feeling beardly.  I itch with enjoyment.

Keeping it trimmed though, gotta be professional.

The Rest Is Silence

This is a collection of pieces I have done over the past few years, and I slowly add to the list.  I think they continue to tell a story, building on one another as they go.  Some are just straight improvisation, so I’ll never be able to replay them as they are.

Please enjoy.