I had an idea last month for a site based around identifying video game streamers while they are in game. If I can identify them in real time then I could have real time fantasy league style betting based around the stats... it was a fun experiment but my techniques ended up not being very practical.

The Idea

In the game League of Legends I could try to find the "yellow health bar" that only your character has. Once I have the coordinates for the yellow health bar I can look above it to see the character name! Boom, with the character name I can look up stats and track all of their games automatically.

The process

For example, watching Krepo's stream here's how I'd go about getting his character name.

Get the stream frame

from livestreamer import Livestreamer

while True:
    session = Livestreamer()
    streams = session.streams('http://www.twitch.tv/%s' % streamer)
    if streams:
        stream = streams['source']

        container = av.open(stream.url)
        video_stream = next(s for s in container.streams if s.type == b'video')

        image = None
        for packet in container.demux(video_stream):
            for frame in packet.decode():
                image = frame.to_image()
                features = process_image(image)

Find yellow health bar

def _find_our_champion(image, search_step=2, draw_on_image=False):
    # should find the yellow bar underneath our hero and then 
    # we can use the top left of that to center our search area
    data = np.asarray(image)
    # remove any alpha channel in the data so we just have (r, g, b)
    data = data[:, :, :3]
    width, height = data.shape[1], data.shape[0]
    character_name_coords = None

    # Cut off the bottom few hundred pixels, not needed
    height -= 200

    for y in xrange(0, height):
        hits_this_row = 0
        for x in xrange(0, width, search_step):
            r, g, b = data[y][x]
            if r > 200 and 160 < g < 230 and 30 < b < 70:
                # really yellow
                hits_this_row += 1
            if hits_this_row > 20:
                # Top left and bottom right
                character_name_coords = (x - 120, y - 35, x + 100, y - 8)
                if draw_on_image:
                    red = (255, 0, 0)
                    draw = ImageDraw.Draw(image)
                    draw.rectangle(character_name_coords, fill=red)
                break
        if character_name_coords:
            break
    return character_name_coords

Process the image

def _ocr_name_box(name_box_image):
    # 0 means load in grayscale
    try:
        gray = cv2.cvtColor(np.array(name_box_image), 0)
    except TypeError:
        gray = cv2.cvtColor(name_box_image, 0)
    ret, gray = cv2.threshold(gray, 160, 255, cv2.THRESH_BINARY)

    # ocr code below

OCR the final processed image

def _ocr_name_box(name_box_image):
    # processing code above

    gray_image = Image.fromarray(gray)
    return pytesseract.image_to_string(gray_image)

OCR results: Krepo

So, it worked in this particular case... this was one of the few that did work.

The results

NOT GOOD!

The idea was super fun to pursue but it doesn't perform very well for a few reasons:

• Streamers can disable showing character name
• The "yellow health bar" specific to your character can be disabled
• What region are players on?
• PyTesseract is pretty cool but Tesseract is pretty cumbersome to use. Hard to train, seems kind of outdated but definitely decent for being free!
• The OCR results were pretty bad although I'm sure with mechanical turk + some training we could get it in a decent state, but still.. names could not be shown at all!

So... it was a fun experiment but not very practical!

You can find all the source here.

Four years ago I had no idea what I was doing in Web Development. Asking dumb questions in IRC all day was how I learned. Scrounging through tutorials trying to make sense of anything. Now I'm the one answering the dumb questions!

I felt like writing this because I've just realized that I now hang out in #django and actually spend most of my time giving back instead of taking. To me that's a huge achievement that deserves some attention, at least.

It's not uncommon that I leave the IRC window up on my 3rd monitor and glance at it occasionally. More and more I'm seeing people ask questions that I have no problem answering... what the hell! Without even realizing it I've transitioned from an entry-level-nobody to kinda-somebody-worth-listening-to.

Being a newb

When I was just getting started I vividly remember being helped by FunkyBob multiple times... I had no idea why he was helping me. Maybe now I'm finally starting to understand why he would helped me, but a few years ago I had no clue. I've bought pizza/beer for a couple people and such, but that's not the right way to give back.

The real way to say thanks would be to, myself, become able to give back to the community.

The transition

I'm not sure when the switch flipped, but my best guess is recently with changes at work I am seeking out more and more ways to be proud of what I do. Instead of just getting the work done, I take pride in taking my time implementing a better solution. Not something perfect, but something I'm proud of.

I've also made horrendous fuck ups. Huge security holes, missed details, and poorly designed features. Though these seem like awful things, this where my growth seems to be happening. By being forced to confront these problems without the option to give up/change course: I am learning a fuck load.

It's regular to get stuck for hours on something simple like forgetting to pass *args, **kwargs to a super method, but in those painful hours I grow and stretch my brain around the problem. When I finally understand just how stupid I am with the small mistake, it's like taking a load off my shoulders.

Sometimes it feels like I have no idea how I'll get to the finish line on a feature. That's getting more rare. My confidence is growing that my weak points are shrinking. In the way I communicate especially.

Good for you!

I know this is kinda lame congratulating myself... but I'm finally getting happier doing web development, damnit! Our job is filled with negativity, set backs, and pressure--so why do we do it? Of course money is there, but that won't keep me around.

What will keep me around is this:

rpkilby: thanks for the help

xcyclist: Yes, that did it. Thanks.

shutupsquare: Hey Thanks, I never even thought to have the items have a fk to the person, perfect, thanks guys.