Hey journalists — here’s why you should learn to make the internets

In case you didn’t know, many news organizations are looking to hire web developers to help create awesome stuff online. And as this list of 19 job openings might imply, news organizations are having a difficult time trying to find people to fill these gigs.

One of the reasons for this difficulty, I believe, is because journalism students are not learning the necessary skills during their college years and internships to meet the minimum requirements of these jobs. Now, I’m not going to blame the schools in this area entirely, as most of the news developers I know are either mostly self-taught or had only a little encouragement from their institution of higher education.

So this is an issue, methinks, of students probably wanting to get more traditional journalistic jobs, from reporter to copy editor to photographer to page designer to whatever. Your standard bread and butter gigs, the ones that are needed to put out that daily (or weekly) piece of paper with ink. That’s great, and at one point in my life that’s all I wanted more than anything.

When I first got into journalism, I decided I was either going to be the next Chuck Klosterman or Bob Woodward. This made sense until I realized that most journalism students were also attempting to head down a similar — or more traditional — journalistic career path. Combine that with The Great Journalism Job Repression of the late 2000s, and you had a lot of people — some of them experts in their field — looking for the same kinds of jobs.

Because my land-grant institution required me to take a few economic classes, I understand how price and demand works. You have a lot of one thing (journalistic talent) and a lowered demand. This means lower prices (salaries) and a lot of unsold goods (unemployment).

I respond to incentives, just like everyone else. Post-graduation employment was a big incentive. So I made a conscious decision to find something in the market that was a scarcity. First it was computer-assisted reporting, because not many people had these skills and I thought it’d set me apart. Then it turned into web development, a field fewer were involved in.

Before I explain the economic incentives more, let me give you some of my back story. I spent my final few years of college focusing on online-oriented journalism, from shooting and editing video and audio to learning most of the basic content management systems. Then I graduated, was hired full-time during an internship at the St. Petersburg Times and was put to work covering the early morning cops shift.

So how and why did I go from a reporter who knocks on doors after murders and shootings to someone who sets up servers and works with databases? For one thing, covering the cops beat is emotionally draining, which is a topic I will focus on in a later essay. But another part is that my background in computer-assisted reporting pushed me to find new ways to present my normally word-focused stories online.

And let’s not deny a major component of my jumping to web development: My ego. Like always, I wanted to do what the cool kids were doing. And at the St. Pete Times, some of the cool people were building PolitiFact, MugShots, HomeTeam and the like. So I started helping out when I could, pitched a few ideas and occasionally got time away from cops reporting and general assignment work to build a few projects.

From there I learned how to program a basic project in Django, which means setting up the databases, making the project talk to a server and then spitting out data on a page. Others would help with the design-side of the projects — I just made the damn things work. Then at a later job with The New York Times Regional Media Group, I got to understand more about the project management aspects of web-oriented work, including dealing with multiple stakeholders, doing quality assurance tests and designing projects according to specifications.

But most importantly, this work was still interesting. Instead of interviewing folks and finding new sources, I was discovering new technologies, finding tools that let me do what words couldn’t. The news application community is also very active on the internet, especially Twitter, quick to answer questions you may have. Some have been more than willing to double-check my code and offer suggestions. Similar groups exist in traditional journalism, of course, but I’ve always liked how the web development community focuses on education and training of one another.

Oh, and let’s not forget that the copy desk has yet to call me at 10:30 p.m. since I moved to web development.

Now, I know this probably isn’t much of an incentive to you, oh youthful journalism students of the world. You’re employed, Andy. Neato. Way to go, etc. But I thought I’d talk about something that you don’t really hear about during your schooling: Salaries.

Let me start off by saying that money isn’t everything, folks. But when you graduate you’ll quickly discover how expensive the real world is. So let’s do some basic math about your expenses. These are the expenses a friend of mine who works at a smaller-sized metro daily pays a month This person has two years of experience. Anything in bold I added or edited:

I split a $900 rent payment with (significant other) every month, and that includes some utilities. We split $80 for cable/Interwebz. Car payment is like $260. Credit card is $200. Electricity is about $35-$40 (split two ways, so $18-$20).  Then there’s groceries (about $100-$150 a month), split two ways. And maybe $100 on gas. And cell phone is $65, car insurance is $80. And I make $900-$930 every two weeks (after taxes and other deductions). Woo.

So, that’s about $1,300ish in monthly expenses on top of $1800ish or so a month. Remember, this doesn’t include other life expenses, such as getting dinner on occasion, going to the bar (as many journalists are want to do) or fixing your car’s transmission if it fails. Or doing anything fun that costs money. Obviously this would make it hard to save up to put a down payment on a new car or house. As my friend says: “Basically, I’m kind of poor, but I’m pretty pleased…theoretically I should be able to save at least $3,000 a year. Not happening.”

Compare that to another friend who does web development in a much more expensive metro area at a much larger news organization. This friend says: “I make between 70-80gs, with 2 years experience. Reporters with the same experience (at my office), make around 50-55.”

Again, this friend lives in a much more expensive city than the previous friend, with the cost of living being about 41 percent higher. But the second friend also chose to take a less traditional path, one that has fewer people with these skills. And as that list of 19 job openings that have yet to be filled indicates, it’s hard to fill these positions. Why? Because not many have the skills to fill them.

So, in summary, journalism folks: It is in your economic interests to learn how to make the internets. Sure, being in one of the traditional roles is pretty neat. Those traditional roles are what fund about 85-90 percent of the profits of most daily newspapers, so they’re what generally fund the salaries of the webby folk.

Journalism is a business, first and foremost. Sure, it’s also a calling. But if we can’t pay for the dedicated women and men who do this important work, then the important work won’t get done. So we’re trying to come up with new revenue streams online so we can start to make enough money to pay for everyone. Even though your career may not go where you originally intended, your work on the news organization’s web site may help fund journalism. Isn’t that important?

I think it is. So take the plunge. Come up with a web project and build it. Go through my Django tutorials. Just do something. You won’t regret it.

A funny thing happened to the St. Petersburg Times

In my morning email blast from the St. Petersburg Times, I saw a column from Sue Carlton that looked interesting. After I read the lede, I didn’t feel like reading any more. Yet again, it was one of those “A funny thing happened (insert whatever your story is about)” ledes.

I would never claim to be anything but a mediocre writer at best, but even I know to stay away from this cliche, in both my ledes and elsewhere in my writing. (Someone, please, search through my archives and prove that I’ve never used this expression. Maybe I have? I am certainly not innocent from using cliches in ledes.)

So that made me wonder. How often does the St. Petersburg Times, my former employer and a great paper that lets its writers shine, let its people get away with using this phrase? I didn’t think that often. I was wrong.

I did a search using their “site search” tool with the phrase “a funny thing happened on” and “a funny thing happened” and I got back 24 stories and blog posts since 2000.  That’s, well, more than a handful. In a couple, the phrase appeared in the headline (one was just the headline).

So I thought I’d post them all here, because I’m snarky like that sometimes.

Why I might go to the Aerosmith-ZZ Top show at Tampa’s Ford Amphitheatre July 11 — Sean Daly, July 26, 2009

Dr. Ozzy is in…and surprisingly lucid? — Sean Daly, July, Jan. 28, 2011

The Mother’s Day Playlist — Sean Daly, May 8, 2008

Ryan (Benjamin) an inspiration — John Cotey, April 29, 2001

Air Supply’s Russell Hitchcock talks ’80s nostalgia and opening for AC/DC — Unknown, Aug. 29 (some year)

Cheerleaders get a taste of success at camp — Eileen Schulte, Sept. 3, 2000

For retailers, Hernando County no longer for sale cheap — Jeff Webb, July 15, 2001

2A-10 baseball: Bishop Mclaughlin hangs on through Tampa Prep’s rally — Rod Gipson, April 30, 2010

As the fuse fizzles — Bill Duryea, Feb. 14, 2003

Sorry, parents, you’re overrated — Robyn E. Blumner, May 14, 2006

A funny thing happened on the way to the exit — Andrew Skerritt, May 14, 2006

In IPO fee game, only banks gain — Editorial, Aug. 22, 2010

Hanging 10 with the Beach Boys’ Mike Love — Sean Daly, Sept. 3, 2009

Strawberry Festival brings big hair, big voices — from Taylor Swift to George Jones — Sean Daly, Feb. 24, 2009

Small plates, big shelves — Chris Sherman, June 27, 2007

A funny thing happened on the way to health — Paul Swider, April 12, 2006 (To be fair, this is only in the headline)

Gerald McCoy meets a fan — in Rwanda — Stephen Holder, March 24, 2011

Everlast talks House of Pain’s reunion, Irish-American culture and playing ‘Jump Around’ live — Jay Cridlin, March 29, 2011

Nice win for Raheem Morris, but Tampa Bay Buccaneers should still give Bill Cowher a call — John Romano, Dec. 28, 2009

Class-size changes not dead yet — Ron Matus, April 21, 2009

Coconuts Comedy Club to throw weeklong party for 25th anniversary — Steve Persall, Feb. 17, 2011

Swimming: No conference crown for these champions — Derek J. LaRiviere, Oct. 7, 2007

Swing voters are indignant, not indecisive — C.T. Bowen, May 16, 2004

By reaching the Final Four, Sherri Coale finishes a turnaround — Antonya English, March 27, 2002

Distant dream real for Riggans — Dave Scheiber, Sept. 8, 2006

Cotey: Welcome to Jimbo show — John C. Cotey, Sept. 27, 2008

One bomb away from losing rights — Robyn Blumner, Sept. 9, 2007

Fed actions credited for higher hopes — Editorial, May 16, 2008

Sunday Journal: Nerves adapt as city’s fingers grab at suburbs — Liz Drayer, April 25, 2010

That pawn is going, going, gone! — Dave Scheiber, Feb. 24, 2003

Trimmed limbs appear to solve Brandon outages — Andrew Meacham, Feb. 29, 2008

Budget cuts are bad, but it could be worse — C.T. Bowen, Dec. 14, 2008

Are teachers underpaid? — Ron Matus, Oct. 17, 2004

***

So a few patterns emerge. I see some names repeated. I won’t name ‘em, you can figure that out yourself. Obviously with the deluge of copy the average reporter is expected to throw out these days — blog posts, 1-2 dailies, a weekender, etc. — I can see how similar turns of phrase would crop up. And especially with blog posts, which may not be as carefully edited by editors.

Oh, and yeah, pointing out this phrase in a newspaper is nothing new. Andrew Beaujon did something similar with the Washington Post for TBD. Anywho, I just found this interesting. Maybe you do, too.

***

Update 1:53 p.m. — I misspelled Dave Scheiber’s name originally in my first post. My apologies, Dave!

Step Eight: Configuring the admin and uploading to S3

We set up our uWSGI and nginx conf files and our server is now running. Now we will set up the admin to add content — one of the totally sweet parts of Django — and then show you how to get your admin files onto Amazon S3.

What sets Django apart from many other out-of-the-box frameworks is that it has a pretty bitchin’ admin built into it. You don’t have to use it, of course. You could just do everything from your terminal, and add information in through the shell. But Django was designed by developers who worked for a newsroom. Thus bits of it had to be understood by regular newsroom folk — editors, reporters, photographers, etc. People who weren’t super web-savvy should be able to use it through some sort of web interface.

Hence setting up the admin. When you’re done, it’ll look like this:

Neat, right? This looks like something your average person who’s ever added a photo on Facebook should be able to work, right? Right.

So, first you need to add a file in your project directory (/opt/django-projects/fires/). Name it admin.py. You can do this by either creating a file on your computer and then uploading it, or if you’re using something like Coda, just right click on the in the area listing files and click New File.

Copy and paste this code inside. I’ll explain afterword how it works:

from django.contrib import admin
from firetracker.fires.models import State, City, Address, Department, Station, Title, Person, StoryLink, Injury, Victim, Fire, Cause

class StateAdmin(admin.ModelAdmin):
    prepopulated_fields = {'name_slug': ('name', ) }
    search_fields = ['name']

class CityAdmin(admin.ModelAdmin):
    prepopulated_fields = {'name_slug': ('name', ) }
    search_fields = ['name']

class AddressAdmin(admin.ModelAdmin):
    prepopulated_fields = {'street_slug': ('street', ) }
    search_fields = ['street']

class DepartmentAdmin(admin.ModelAdmin):
    prepopulated_fields = {'name_slug': ('name', ) }
    search_fields = ['name']

class StationAdmin(admin.ModelAdmin):
    prepopulated_fields = {'name_slug': ('name', ) }
    search_fields = ['name']

class TitleAdmin(admin.ModelAdmin):
    prepopulated_fields = {'title_slug': ('title', ) }
    search_fields = ['title']

class PersonAdmin(admin.ModelAdmin):
    prepopulated_fields = {'name_slug': ('first_name', 'last_name' ) }
    search_fields = ['name']

class InjuryAdmin(admin.ModelAdmin):
    prepopulated_fields = {'injury_slug': ('injury', ) }
    search_fields = ['injury']

class CauseAdmin(admin.ModelAdmin):
    prepopulated_fields = {'type_slug': ('type', ) }
    search_fields = ['type']

admin.site.register(State, StateAdmin)
admin.site.register(City, CityAdmin)
admin.site.register(Address, AddressAdmin)
admin.site.register(Department, DepartmentAdmin)
admin.site.register(Station, StationAdmin)
admin.site.register(Title, TitleAdmin)
admin.site.register(Person, PersonAdmin)
admin.site.register(StoryLink)
admin.site.register(Injury, InjuryAdmin)
admin.site.register(Victim)
admin.site.register(Cause, CauseAdmin)
admin.site.register(Fire)

WHAT IS THIS SHIT?

So, if you look at this and compare it with your models.py file, you’ll see a lot of things overlapping. First, you need to pull in everything from your models.py file. This line tells Django to import its admin bits:

from django.contrib import admin

The next line pulls in the various classes from your models.py file. This allows you to later use them in this admin.py file.

Line 4-6 is the basic way we’re setting up everything in the admin. So you need to make a class for the State class. Thus StateAdmin. Line 5 is telling you to pre-populate the name_slug field (which you have in your models.py file, remember?) with whatever is typed in the name slug. You’ll see this in action once everything is live. Then search_fields allows you to include a field to search by in the admin.

Line 40 is where you basically sync stuff up. You’re sort of saying “Hey, State and StateAdmin, make friends!” So you need one of these for each of the previous admin classes you created above.

UPLOADING SHIT TO S3

Now we need to upload some files to S3. Why, do you ask? Because the admin uses some CSS, Javascript and styling deals. Well, why don’t we just put that on the instance we created and are hosting the Django stuff on? Because that would require us to also set up a server to run media files. That would mean more hits to your server, more traffic, more load, etc. And as media files in general — photos, audio, video, etc. — can be large, that means it would also push up your Amazon bill, as they charge for traffic leaving an instance after 1 gigabyte.

Also, I don’t know how to set up a media server that isn’t for PHP. SO THERE.

First, we are going to download a copy of Django onto our harddrive and then pull out the media bits. Go here to download a copy. Make sure it’s your current version — if you have Django 1.2.3 running, get that. If it’s 1.3, download that version. After it’s downloaded, unzip it and look inside. Go to the unzipped file, then this path /django/contrib/admin/media/

Inside of this are all the files you’re going to want to upload to your S3 account. This is sort of an annoying process, and I’m sure there’s a quicker way, but I’m kind of dumb so I end up doing things the longer way. (AKA hey anyone out there reading this, if you have a better way to do this, leave a comment or let me know, and I’ll include it in here and give you props).

So in the end, you’ll want to have uploaded to S3 a css, img and js folder, with all the files in each individual folder, and then each individual folder will have its subsequent subfolders and files. Like I said, this will be annoying and tedious.

To get this set up, login with the account you created earlier to AWS. Click on S3 once you’re logged in. You will now create a bucket. I suggest you either name it after your organization — dailynewstribuneglobecompany — or something. Or name it after your project. Whatever. Once you do that, you need to create a folder named “admin.” Inside of that, add three more folders named css, img and js. And from there, you will upload each individual file, create new subfolders, etc., so it matches the file on your computer. Do that shit like I said you should do up there.

Once that’s done, go back to the main bucket directory. Right click on the admin folder, and then click Make Public. This will make it so people can look at your files without the various AWS credentials.

Now, to test that that worked, go to the equivalent URL that looks like this, only instead of dailynewstribuneglobecompany it’s whatever you threw up there: https://s3.amazonaws.com/dailynewstribuneglobecompany/admin/img/admin/arrow-down.gif

You should be able to see a little arrow appear in the screen of your top left browser. If you can see that, voila, your admin shit was uploaded.

Once that’s done, you will need to edit your settings.py file, like we did in Step Five.

Change line 51 that to:

ADMIN_MEDIA_PREFIX = 'http://s3.amazonaws.com/dailynewstribuneglobecompany/admin/'

This tells Django to look for the admin stylesheets here. Now that your project is live at 123.123.123.123/firestarter/, go there and then add /admin/ to the end of the URL. You can now add in the username and password you entered when you originally syncdb’d back in Step Five.

Once you’re logged in, you should see the basic admin screen that we showed earlier. In my next post, I’ll discuss adding content.

Step Seven: Configuring uWSGI and nginx

Now that we’ve set up the views and whatnot, it’s time to make the damn server work. This is probably the part that is the most difficult, in my opinion, as it’s the difference between a project being an idea on your laptop and being something that’s live and in the world.

After you read this, it may help to watch this presentation by Jacob Kaplan-Moss, one of the original creators of Django and an all-around coding badass. It explains deployment much better than I could, and while it’s super duper technical, it does go through some of what we’re doing here. It can’t hurt to learn more, right?

Anywho, if you remember in Step Two, we added a few directories, such as /opt/run/ and others. We’re using some of the basics that my pal Jeremy Bowers set up in his blog post on uWSGI and nginx.

What’s nginx and uWSGI, you ask? Nginx is, if you remember from my analogy in Step One, akin to the bartender getting you a beer. UWSGI would be more like if the bartender had to understand German in order to find the right beer. It acts sort of as a translator between your app and the server software.

Now, use your terminal and public key and ssh in to your instance. We will now get and install nginx and uWSGI. Do each line one at a time.

cd /opt/
wget http://projects.unbit.it/downloads/uwsgi-0.9.6.4.tar.gz
wget http://nginx.org/download/nginx-0.8.51.tar.gz
tar -xzf nginx-0.8.51.tar.gz
tar -xzf uwsgi-0.9.6.4.tar.gz

Woohoo. We have downloaded stuff. Aren’t we fancy? (Yes.) Now you need to compile and install uWSGI and nginx. It may look scary, but don’t worry. Enter each line one at a time, as per usual:

cd uwsgi-0.9.6.4
make
cp uwsgi /usr/sbin

Not that hard, right? Now this bit’s a little harder. But don’t worry, you can just copy and paste stuff and it’ll all work out:

cd ../nginx-0.8.51/
touch /opt/run/nginx.pid
touch /opt/lock/nginx.lock
touch /opt/log/nginx/error.log

Now copy these lines and paste them into the terminal then hit enter:

./configure --conf-path=/etc/nginx/nginx.conf --error-log-path=/opt/log/nginx/error.log --pid-path=/opt/run/nginx.pid --lock-path=/opt/lock/nginx.lock --sbin-path=/usr/sbin\ --without-http-cache

Now:

make
make install

What did we just do? We told nginx where to look for its configuration file, we told it where to put its log file, and a bunch of other wonderfulness. Next we need to do some more fun configuration.

SETTING UP UWSGI

Go into your project directory, which should be at /opt/django-projects/firetracker/ and then we will make a few files:

mkdir uwsgi
touch uwsgi/__init__.py
touch uwsgi/wsgi_app.py

This is where Mr. Bowers would have you edit code in the terminal. But guess what? That always screws me up. So we’re just going to use FTP to edit that wsgi_app.py file. Go to the /opt/django-projects/firetracker/uwsgi/ directory and open the wsgi_app.py file using whatever FTP thing I told you to use (or whatever you’re using), and then paste this inside:

#! /usr/bin/env python
import sys, os, django.core.handlers.wsgi
sys.path.append('/opt/django-projects/')
os.environ['DJANGO_SETTINGS_MODULE'] = 'firetracker.settings'
application = django.core.handlers.wsgi.WSGIHandler()

Save that bad boy. Now we’re going to make a configuration file for this project that’ll help you control uWSGI. Do this:

cd /etc/init/
touch firetracker.conf

This file will allow you to quickly restart your app, using a simple command like “sudo service firetracker restart.” You’ll need to do that in the future if you ever change a view, model or admin file. So it’s good to have a shortcut. Now open up the file at /etc/init/firetracker.conf and throw this information inside:

description "uWSGI server for ProjectFiretracker"
start on runlevel [2345]stop on runlevel [!2345]
respawn
exec /usr/sbin/uwsgi
--socket /opt/run/firetracker.sock
--chmod-socket
--module wsgi_app
--pythonpath /opt/django-projects/firetracker/uwsgi -p 8

Jeremy’s post explains more about what this means. But hey, this is just meant to get you started. You can learn more about how the hell this all works later. Next thing you need to do is run this command to let the server know it has a new toy:

sudo initctl reload-configuration

And then do this to start your uWSGI instance:

sudo service firetracker start

You can also replace start with stop, or restart, and it’ll do what you think it should do. Just remember that any change you make to most of the core Django files, you’ll need to restart firetracker by ‘sudo service firetracker restart’ so it pops in the changes. Also remember that this needs to be running as well as nginx or else nothing will show up. Speaking of which, let’s set that up.

SETTING UP NGINX

NOTE: Some pals have had errors with the nginx.conf later described in this tutorial. If you do, e-mail me at my website address at the email service Google happens to use.

The last thing we need to do is edit our nginx configuration file so it can talk to the uWSGI instance you just created. Using FTP, open up this file /etc/nginx/nginx.conf. Copy and paste this code inside and then save:

user www-data;
worker_processes 1;
error_log /opt/log/nginx.log;
pid /opt/run/nginx.pid;
events {
worker_connections 1024;
use epoll;
}
http {

include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
keepalive_timeout 65;
proxy_read_timeout 200;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
gzip on;
gzip_min_length 1000;
gzip_proxied any;
gzip_types text/plain text/css text/xml
application/x-javascript application/xml
application/atom+xml text/javascript;
proxy_next_upstream error;

server {
listen 80;
server_name your-long-ec2-instance-name-goes-here 123.123.123.123;
client_max_body_size 50M;
root /var/www/ht2;
location / {
uwsgi_pass unix://opt/run/firestarter.sock;
include uwsgi_params;
}
}

The only bit you should really notice is the the line that starts with server_name. After that you need to put your long EC2 name, which you can find in your AWS administrative panel on your EC2 page. Click on your running instance and then in the bottom window that pops up, look for “Public DNS.” This should look like ec2-123-123-123-123.compute-1.amazonaws.com or something. Copy that and paste it into the area after server name. And then take the numbers in the 123-123-123-123 — which is your IP address — and paste them after the instance public DNS (with a space in between).

Now run this:

sudo service nginx start

And voila! You have uWSGI and nginx running, listening on port 80. You won’t need to “sudo service nginx restart” unless you change something in the .conf file, which you won’t need to do unless you add another project in the future. Don’t worry about that for now.

Go to 123.123.123.123/firetracker/ It should spit out some basic information. But nothing will load because you have no data entered! We will do that in the next post. Now let’s move on to Step Eight, which sets up the admin.

Step Six: The Views and Templates

Now that the database is set up, it’s time to make our views and templates work. In essence, the views talk to the models, or the database. The templates then talk to the views and spit out the content. It’s super awesome.

So, let’s edit our views.py file. That will be located in /opt/django-projects/firetracker/fires/

Here’s the code to copy and paste into the views.py and then save:

from firetracker.fires.models import State, City, Address, Department, Station, Title, Person, StoryLink, Injury, Victim, Fire, Cause

from django.shortcuts import render_to_response, redirect, get_list_or_404, get_object_or_404
import urllib
from django.conf import settings

def index(request):
    fires = Fire.objects.all().order_by('date')[:5]
    return render_to_response('index.html', {
        'fires': fires,
    })

def fire(request, address, un_id):
    fire = get_object_or_404(Fire, location__street_slug=address, id=un_id)
    return render_to_response('fire_detail.html', {
        'fire': fire,
    })

def person(request, slug, un_id):
    person = get_object_or_404(Person, name_slug=slug, id=un_id)
    return render_to_response('person_detail.html', {
        'person': person,
    })

The first line brings in information from your models.py file, pulling in all of the classes we plan on using. The next few lines are to say “hey this is a views.py file!”

Line 7 is where we set up up our first page. This is the index, which will be our main /firetracker/ page in the URL. We will set up the URLs later in this post. But for now, let me explain some of the basics.

Line 8 makes a variable called “fires,” and we’re saying that fires should find every “Fire” in the database (Fire is a class we made in the models.py file, remember?). The .objects.all() part is saying “Find everything and spit it out.” The .order_by(‘date’) is telling it to list everything by date, which is something each Fire contains. Then the [:5] means “Show five.” If you wanted to show 15, it’d be [:15]. Then we tell it on line 9 to bring the information and spit it onto a template called index.html, which we will create later in this post.

Lastly, on line 10 we say that to allow the variable “fires” to be the same as the “fires” we defined on line 8. This means “fires” can be something we will call on the index.html page. Trust me, this will make sense later.

The next page, starting at line 13, is more complicated. This is for each individual fire page. You may notice that we define fire — def fire — and then ask for (request, address, un_id). So the request we can ignore. But the address and un_id are also variables from the urls.py file (this will all make more sense later). Line 14 is similar to line 8, except we’re doing a few extra things.

First, we tell it to get_object_or_404. This means if it doesn’t find anything matching in the database, show the 404.html page. The bits in the parenthesis — (Fire, location__street_slug=address, id=un_id) — is trying to make a match. It’s saying, “Okay, look in the Fire class. See if you can find something that has a location slug that is equal to the ‘address’ part of the url. Also, the Fire’s unique ID has to match the ‘id’ part of the url.”

For instance, the url will look something like /firetracker/fire/2/500-e-10th-street/ — 2 is the unique ID of each fire, and 500-e-10th-street is a web-made version of 500 E. 10th Street. In the admin file we create later, we’ll make something that automatically makes slug files, so don’t worry about this right now.

And then on line 19, we are making the individual person page. Similar as the individual fire page, but we’re matching the name and each person’s unique ID.

EDITING THE URLS.PY FILE

Now we gotta edit the urls.py file, which is located in the same directory as your views — /opt/django-projects/firetracker/fires/ — so open that up in your FTP client.

Go to this link and copy and paste the code (this isn’t posting into WordPress correctly, FYI). This is where we are defining our url structure. The index page will show at 123.123.123.123/firetracker, for instance (where 123.123.123.123 is the IP address of your AWS instance).

Line 7 has a bit that says firetracker.fires.views.index. That’s telling it “Hey, go to the views file for fires, then find the bit defined as ‘index.’ Use that for this url.” Line 8 and 9 are the more fun bits, as they are the ones that are more interacting with the views. If you see, there’s a un_id and address part in line 8. That’s corresponding to lines 13 and 14 in the views.py file, where it’s trying to make the URL by first finding if that “fire” exists, and then it’s spitting it out.

I’m sucking at explaining this, so, I apologize. But trust me. Shit will work. You will be happy.

MAKING THE TEMPLATES

Now that we’ve got the views.py and the urls.py made, time for some templates. As you may recall from the views.py, we have three templates: the index, the individual fire page and the individual person page. Using your FTP program, make a new folder called “templates” in your /opt/django-projects/firetracker/fires/ directory.

Inside of that, create three new files. Can you guess what they are based on the views.py?

YOU’RE RIGHT. You rule. They are, indeed, fire_detail.html, index.html and person_detail.html. You’re a fantastic human being, and I’m glad to know you.

Go to each of the following links and then copy and paste them into each of their corresponding files.

index.html

person_detail.html

fire_detail.html

Awesome. We now have our views. They’re not the prettiest ever, but they get the basic point across. Next we will configure the server to display this stuff.

Shipping up to Boston

Today I accepted an offer to join the Boston Globe as a newsroom web developer. I’m super excited to be joining such a great newspaper, one that has a long and distinguished reputation. I feel like I won the lottery.

So that brings me to the sad part about this announcement. I am also leaving The New York Times Regional Media Group, a job I’ve enjoyed immensely. The work has been fun and worthwhile, and I’ve been a happy camper since I started. They’ve given me the latitude to get projects done generally in the best manner I saw fit. They gave me the trust and responsibility to deal with important projects. And most importantly, they pushed me and my fledgling programming skills, allowing me to get stronger every day with each new project.

The NYTRMG is a great place to work. My boss, my coworkers on the dev team, the product managers, everyone here has such a great attitude and comes to work ready to tackle whatever obstacles are in the way. These are quite talented folks, and I was lucky enough to have met them and been able to learn from them. If you ever get a chance, apply for a job here. You won’t regret it.

Also, I love the Tampa Bay area, especially St. Petersburg. This is a wonderful place to be a journalist, and a great place to live, especially if you’re a 20-something who likes movies, riding his or her bike and occasionally — just occasionally — imbibing on an occasion or four.

That’s why this is a sad and happy decision for me. I love this area and my job rocks. So I’m taking a big leap, moving to a Real Big City, starting a job at this paper that’s always been on my radar for its overall awesomeness and making a whole group of friends.

I’m a Nebraskan, so I’m sure I’ll be able to make some new pals, as it’s in our hearty Midwestern nature. And I’m Irish. I think my blood requires me to spend some time living in Boston for awhile. It’s the law, I think.

So, that’s that. Packing boxes and moving in a few weeks. But for now, I can’t wait for this adventure to begin. Wish me luck. In the meantime, you should play this song really loud and think of me.

Cheers.

Thoughts on Remembering the Storm’s design and implementation: Part Three

In part one I discussed why. In part two I discussed the how. Now I shall discuss more of the how and the how did it go.

So when I got home, I started to work on getting the site live, at least on the IP of the Amazon instance. I had previously switched my Recruits project to being at the ipname/recruits, which is what the Varnish instance was pointing to and caching. Thankfully, good pal and well-known lover of fancy food Jeremy Bowers had once written a great tutorial on using uWSGI and nginx, which is what my Django server was already running (and I had since forgotten how it all worked). And he was online and helped me out when a few things, uh, didn’t work.

Then I had to fix some stuff in Varnish. This is where Michael Strickland, boy genius and Jesse Eisenberg’s long-lost brother, had a good idea. He told me how to set up my nginx.conf file better than Jeremy did, and made me abandon my super-difficult-and-extended idea of how to set up Varnish (which may be, now that I have talked to others, maybe not such a bad idea to begin with). I’ll go more into detail in the intense nerdery in a later post.

So, I sudo service restarted and bam. It worked. EVERYTHING WAS ON THE SCREEN! Whenever such things happen I usually almost cry, because I constantly fear that maybe I’m a con artist and I don’t know what I’m doing. But it worked, with some help from my pals. Woohoo!

Next I made sure the admin was working and set it up so it’d be easier to use for our various newspapers. I set them up with usernames, added in some fake information for placeholder and got the correct Omniture tags from Andy Rhinehart, one of my coworkers. Metrics are important, as we wanted to know how well/not well the app was doing. Otherwise, how could we learn from it?

The Next Day

I went to bed around 4 a.m., slept a few hours then went to work. It was Friday. First thing I did was write an email to the other newspapers to get them abreast of where we were at. I had started developing it Thursday afternoon and then realized I had to ask them if it was okay. So I emailed and asked, and they all agreed and bought into the idea. This is important: If our partners didn’t want to work on it, it surely wouldn’t have been a success, in my opinion. I sent them their login information for the admin and told them they could start adding names when they got a chance.

When my coworker who did the design, Yinnette Olivo, got into work, I showed her where we were at. We came up with a quick list of things she had to do and I had to do and we got to work. I still had to fix a few bits of the urls.py file — the part that sets up the url structure and connects to the views — as some of my urls weren’t working. She made some jquery fixes and made it prettier in general.

By late morning, the other newspapers had already started to add names and information, so we had something to work with while double-checking the design. Then after forgetting to eat lunch, we were more or less ready to go. I sent the working address to a few friends who aren’t web savvy to ask their opinion. Some of them critiqued the copy we used and had better suggestions. I liked them, so I changed it.

Before we launched, I realized we had some text different in different spots. I wrote a new include and put it in all of the templates. I would at least follow the Don’t Repeat Yourself philosophy a little bit, you know, just for shits and giggles.

The Launch and Some Thoughts

Then at 1:07 p.m., we launched with about 10 names loaded. It had been about 25 hours since I came up with the concept. This was the fastest I’d ever built anything. I mostly did this to prove a point, like I tend to do many things. Breaking news and web development can and should go hand in hand. I, along with many other people, have proved it. This guy is goddamn wrong.

I did this with the help of a few pals, but it was mostly me and a designer. If this is all you need to make fast projects like this, we should be doing it more often as an industry. The only excuse, in my opinion, is that some people fear giving such authority over server and data architecture to people who aren’t, well, traditional developers. I think this is a mistake, and the media organizations who seem to be doing great work are the ones who’ve broken down this barrier.

That being said, here’s what I learned:

1. Your original idea for how an app should be used may be wrong

I built this with the idea that we would start adding information about every state’s victim. Slowly, though, I realized we would just be focusing on Alabama. Perhaps in the future we will add more names, but right now the focus is on Alabama. Because of this, I changed some of the templates to get rid of a “See All States” tab.

2. Facebook comments > Django comments

I had originally planned on setting up forms and doing a bunch of Django comments muckery. I had never used them before, but I thought they were neat and totally worth using. Matt Wynn told me I should just use Facebook. And we did. It was much easier to deploy and, frankly, has helped to drive a lot of traffic, according to our metrics.

3. Check your ad code and metric code

Multiple times I screwed up adding the right Omniture code. I did it multiple times. I also didn’t have the right advertising code until Monday. Why? Because I was dumb and didn’t ask our ad department or another coworker to get that stuff right in the beginning. This made it harder to track the metrics. I now know this should be the first step when setting up a new project when you get to the actual building phase.

4. If you’re stuck, ask for help

Like I’ve pointed out multiple times, I asked my pals for help on occasion. Sometimes I just threw ideas past them to see what they thought. Other times I showed them code and they offered suggestions. This is why you should be reaching out to your fellow geeks and nerds, making good connections with them and being helpful to one another. Also, Twitter is a good resource for throwing out OMG HALP on occasion.

5. Reuse your old code

Part of the reason I was able to build this so quick is that I documented in my blog how to set up a basic app, and I also had lots of old code I could reuse. I know this is probably a no-brainer, but thought it worth noting.

6. More photos

The people who have photos have, by far, the most Facebook comments. People are more apt to click on photos than just placeholder images. If you’re setting up something similar, remember to try and always get photos of the actual people. It has a bigger impact, and people are more drawn to it. Our newspapers were calling nursing homes and checking social media for photos. We also were sure to include information on how to contact the newspaper to send us photos or let us know about any data that may be wrong.

7. Don’t do something fancy when you can just make it work // Code whatcha know

I probably could’ve attempted to set this up as my first Ruby on Rails project. Maybe it would’ve been slick to use Mongo or node.js. But I didn’t have time to be fancy, so I used Django, an MVC-styled framework that I know. What is the time to be fancy? When you have longer deadlines and can try out new stuff. Or when you spend your free time — like many of us do — learning new toys so you can use them when the time comes. I could have set up the Django sites framework, but I didn’t because I didn’t want to spend the time learning it on the fly when I knew I had another way that could work. Probably bad programming decision, but a good workflow decision.

8. We should be doing more projects like this

I can’t disclose traffic numbers, but suffice to say this thing got a lot of traffic. Personally, think it was good journalism, and it certainly is better than just a list of people’s names. I was able to do this on two Amazon Web Services small instances using all open source technology. If my dumbass can figure out how to do this stuff, you probably can, too. I’d be even willing to help you. Also, this was a good business decision. It helped to increase traffic to our site and get users engaged. It promoted our newspapers’ brands on social media when people posted comments, and it also showed up on some blogs in Alabama. So it’s good journalism and it’s good business. Why aren’t we doing more of this, especially when the costs — other than personnel — are pretty minimal to get started?

***

Well, that’s that. I wanted to walk people through my process, so I hope I accomplished that. This was probably the project I’ve felt the most connected to in my life. Many times, especially Friday night when I added 87 names and continued to see people who appeared to be relatives — or, in one case, an entire family — I did get pretty emotional. I just wanted to do it justice. Hopefully I did.

Let me know what you think in the comments. Thanks for reading.

Thoughts on Remembering the Storm’s design and implementation: Part Two

Following my basic thoughts behind the basic app setup, now I had to build the damn thing. On Thursday I didn’t have my laptop with me, so I did everything from noon to six on a Windows computer, using Notepad++, FileZilla and PuTTY (which I discuss using a bit in this post.)

First, I opened up a file and started writing the models file. I was able to copy a lot of the code from recruits.gatorsports.com and reuse it. I thought of the basic data we would get: Location, Name, Date of Birth, Age, Photo, Cause of Death, Link to a Story and a bio. When thinking of the data I would be able to collect, I thought of a few of the sites that’ve been built to track members of the armed forces who’ve died in war, most notably the project Matt Wynn and Ben Vankat built at the Omaha World-Herald.

Look familiar to our finished app? Yeah. Because it’s a great idea on how to structure the info. Props to them, and thanks for the idea, fellas.

So I wrote out the models in about half an hour and then sketched out a quick wireframe for the pages on a few sheets of paper. I’ll take photos and post them tomorrow, but the basic idea I had my coworker Yinnette Olivo turned into what you see now. The main page would have a simple nav bar and then photos that would brighten and have a name pop up when you rolled over them. Then the individual pages would list the collected data I had in the models.

The designer got out of a meeting, I showed her my wireframes. She laughed at how crappy my drawing is and then she got to work while I dealt with the views. With my projects, I always like them to have a focal point. With a project I built called MyLawmaker, it was the search bar where you entered your address. With this project, it would be the photos.

After handing off my ideas to her, it led to me realizing a problem. We had two websites that would need this — gadsdentimes.com and tuscaloosanews.com. Because I’m not super awesome with Django, I had never used the sites framework that Django has. And because I’m stubborn, I wasn’t about to learn on the fly. So on the strong suggestion from Ryan Pitts, I wrote a bit in each view that looks for the sitename, and then includes the right header or footer based on it. Snazzy, and totally against the Don’t Repeat Yourself principal.

But as Pitts said, this was more important. So repeat myself a lot I did. And it worked, so I don’t care as much.

By about this time, I had the models, views and admin up. I added some fake data and made sure the photos would upload to Amazon S3, which they did. I discuss how to do that in your settings.py file in this post. I had set up everything so far on my live server (yup, no production server, because I’m dumb), including setting up the database (which I discuss in this post, as well), realizing on the fly I had to reconfigure some of the urls.py in my Recruits project, and then change the settings on Varnish.

I was stupid and had originally set up Varnish in a way that wasn’t exactly conducive to adding new projects. So, don’t do that. Always set up Varnish and your production server to be able to add new projects quickly, people.

By this time, my coworker had the basic templates made on her end. She left work and I fixed a few odds and ends.

We normally do a projectname.sitename.com type of structure with our projects, as opposed to sitename.com/projectname. Our DNS stuff for these two papers is controlled by our CMS provider. So before I left work I put in a high priority ticket to get our DNS settings to point stormvictims.gadsdentimes.com and stormvictins.tuscaloosanews.com to the Varnish server. This was finished by the time I got home.

I ate a quick dinner and thought more about the design, setting up the urls.py better and getting the thing ready for implementation, which I will discuss in my next post along with my final thoughts.

Thoughts on Remembering the Storm’s design and implementation: Part One

At 1:07 p.m. on April 29, I launched a news application faster than I dreamed possible: In almost 24 hours. Now that I have a few days to look back at the process of designing it, I thought it would be beneficial to walk through what I learned and what could perhaps help you in a similar breaking news situation.

But first, the background. As many know, on Wednesday night, April 27, a massive and brutal storm system rolled through the South. Tornadoes ravaged the area, with Alabama was one of the hardest hit states. We had papers in Alabama, in Gadsden and Tuscaloosa. Friends were visiting from out of town so I didn’t spend much time Wednesday learning about the destruction, other than watching a few videos before going to bed.

When I got into work Thursday, the headline at nytimes.com punched me in the face. Already scores were confirmed dead, with officials estimating that Alabama’s were only going to rise.

My first reaction was to start building some Google Forms, so people could enter in the names of their loved ones, damage they were seeing and their reactions. You can see some of that here, here and here. The Tuscaloosa News printed a lot of what people filled out in the forms, and I was told families were able to connect with one another. I also set up a crowdmap at alabamastorm.crowdmap.com, but I will talk about that in a later post.

As I was microwaving my lunch, I came across a post on another newspaper website. It was a list of the identified victims so far. All it had were county/city headings and then names. Sometimes ages. That was it. I saw this and it reminded me of some research I did during my undergraduate days, when I looked at old newspaper archives. During World War II, many casualty lists were just that, simple lists, such as “Pfc. Bob Smith of Dakota City, Neb, 19.”

When I saw that newspaper’s list, it made me a bit emotional. All I could think of was, “Is this how I would want my family to find my name?”

No. No it wasn’t. And it shouldn’t be that way. It didn’t have to be that way. This wasn’t a listing of salary data for some government employees. These names were family members, college classmates, friends, wives and neighbors. Not that every time we throw up some database with people’s names they weren’t somehow worth less. But I don’t know. This struck a chord with me.

I was kind of mad. And I was going to do something about it.

Why I Chose Django

Because time was a factor, I decided I would build it in Django. Here’s why:

1. Speed

I wanted to be able to work on the backend bits while a designer was making it look pretty. Django, unlike PHP, would allow for me to do that.

2. Stability

Django’s pretty slick when it comes to ORM mapping and whatnot, and I assumed this site might get a few page views. So I didn’t want to deal with building it in PHP, getting the SQL all pretty and whatnot.

3. The Admin interface

I didn’t want to have to build a separate admin in PHP, like we’ve done for other projects at work. I knew I would have the staffs of two newspapers and my office working on this, and I wanted an easy-to-use admin interface. Django’s is awesome, and I had used it on a previous project – recruits.gatorsports.com — and their staff was able to figure it out after a little explanation.

4. Had the infrastructure already set up

The main reason I was able to launch this project so fast is because I had the infrastructure for it. We use Amazon Web Services to host our projects for the most part. I already had one small Ubuntu instance set up with Django and Postgres running a live project — the aforementioned recruits.gatorsports.com — and a small Ubuntu instance running Varnish for caching. I knew it was able to take a good deal of traffic as it previously had on Signing Day.

So, I was going to build it in Django with Postgres on AWS, and then use Varnish to cache it. In my next post I’ll discuss some of the problems I ran into while building and designing it. In my last post, I’ll talk about deployment and what I think was done right and wrong.

Ira Glass took a paycut, and so did other public radio hosts

I will take a break from my normal nerd blogging and would like to talk about the salaries of people you might know. OMG SCANDALOUS!

A chat with a friend about IRS forms for news non-profits made me wonder about how much someone like Ira Glass makes. I assumed he’d do quite well, as he is incredibly talented and his show is carried across hundreds of stations.

It’s not unusual for people in high-level journalism jobs who have decades of experience to make a good deal of money, as they often work more than your usual 40 hours a week and they usually have skills and talents few others possess.

So I went to guidestar.org, a site that tracks, among other things, IRS form 990s, which non-profits are legally required to file once a year (unless they get an extension, and then it can be even later). The most recent one they had was from the 2008-09 year. I was quite surprised to see our favorite bespectacled radio host made $159,371, with $11,234 in other compensation (this can mean for benefits, 401k, travel reimbursements [I think], etc.).

That seemed somewhat low. I checked WHYY, the public radio station that produces “Fresh Air,” and their 990s show Terry Gross made $225,303 in 2008-09.

So I looked further, checking National Public Radio’s tax forms for 2008-09, the most recent ones available. Steven Innskeep, one of the hosts of NPR’s Morning Edition, made $309,225. His co-host, Renee Montagne, made $297,510. All Things Considered hosts Michele Norris and Robert Siegel made $286,144 and $295,010, respectively.

Of course, Glass doesn’t do a daily show, like the other public radio personalities. But how does his salary of $159,371 compare with the rest of his staff? Producer Alex Blumberg, who in addition to his work on “This American Life” helps out with NPR’s Planet Money blog and podcast, made $143,020 in 2008-09. Julie Snyder was at $137,742 that year. So he’s not that far above his other coworkers, which is interesting.

Then I checked the previous years. In 2007-08, Glass made $216,000, Snyder made $155,521, and Blumberg made $147,383. Looks like Glass and the rest took some paycuts. Glass took a 26 percent paycut, Blumberg a 3 percent paycut and Snyder took almost a 13 percent paycut. Wow. I’m willing to bet that Glass and Snyder cut their paychecks higher than others, just so less staff would need to be laid off. That’s just a guess.

Meanwhile, NPR’s 2007-08 tax forms only lists salaries for Siegel, Montagne and Inskeep (of those I previously mentioned). Siegel made $319,300 that year, so he took an 11.5 percent paycut. Montagne made $360,826, so her salary dropped $21 percent. And Inskeep’s salary dropped 12.4 percent to $353,390.

Anywho, I just found this information to be interesting. And to point out how awesome IRS 990s are.

Update 11:10ish a.m.

Apparently this is NPR’s 40th anniversary today. Didn’t plan for that. BUT I STILL LOVE YOU GUYS AND EVERYTHING YOU DO!

Update  11:36 a.m.

Also, I checked Public Radio International’s 990s and didn’t find any listings of Ira Glass, just to see if they were also compensating him. And, of course, let’s not forget that he also made that awesome TV show, so he maybe made a few bucks off that. Glass also gives lectures, talks, helps edit books, etc., which I’m sure compensate him somewhat.

But talent costs money, folks. I think that’s a good take away from this.

Update 11:50 a.m.

According to American Public Media’s tax records, Kai Ryssdal, host of Marketplace, made $235,895 in 2008-09.  He made $167,590 in 2007-08, those tax records show. The reason for the jump? Either he was also being compensated by some records I didn’t find in 2007-08, or maybe his bosses realized how awesome his name was and they had to give him more money.

Update 1:32 p.m.

Hey people from Poynter.org and Romenesko! Waddup?