Hello, and welcome to yet another worked exercise.
I'm the author of the book Python for Informatics: Exploring Information,
as well as your host for this particular worked exercise.
As always, the book, this audio and video,
and these slides are copyright Creative Commons Attribution, and
I hope you find exciting and interesting ways to
reuse them and re-purpose them and add to them.
So pythonlearn.com is my support website for the Python for Informatics book.
And if you, hopefully by now you've got all this
worked out, but it does teach you how to get started.
So that, how all the things that you need to know,
to edit files and use the command line to run Python programs you've got figured out.
Okay?
So our program for today is from the list chapter.
And our program is actually, we're going to debug a program
rather than just writing a program from scratch.
And so what our, our task is, in this program is we're looking
for words that start with from, looking for lines that start with from.
And on the lines that start with from, we're going to pull out the
day of the week that this particular
email message was sent. Saturday, Friday, whatever.
And so the structure of the program is pretty standard.
We're going to open a file at the top.
We're going to loop through the file.
Strip the newlines at the end, right?
There's a little newline at the end of each line.
And then we're going to use split to split it into words,
which means that'll make this the zero word, the one word,
and the two word.
And then we're going to check the zero word to see if it's from,
and if the zero word's not from we're uninterested in that line so we're going
to skip back up. And if it is good, if we wfind a line
we're interested in, we're going to skip the majority of the lines in this file.
Less than one in 100 of these lines actually
have from in the, as the first word.
And then the second word is what we're interested in
so we're going to print the, the day of the week.
So this program as we type it in is going to have a
traceback and we go like, you know, as soon as you see traceback
you immediately are drawn to the word traceback and maybe you've gotten to
the point where you start to read the why, list index out of range.
And then, you know, it says line five, so that's kind of helpful.
But the first thing to not get distracted about is, is this.
Our program actually ran a little bit.
So when you look at a traceback, look right above it and make sure, and
maybe you won't see any output, but maybe
your program will have partially run, and so
don't immediately assume the program is totally broken. So this
is actually successful output of one of the many from lines.
And it, it dies later.
And so, as we debug it, we'll sort of come up with techniques to sort of figure out
how much your program has done before it died,
because that might be an important, important question to answer.
How far did your program get, because if it dies before the first line, or on
the first line, that's something different than if
it goes like 300 lines and then dies.
If it goes a long ways, 300 lines, and then dies,
maybe there's something subtle or weird about the line it's dying on,
rather than just your program is
just like totally broken.
So you got, look for something where your program actually partially worked,
and helps you kind of narrow down your suspicions as
to what might be wrong with this.
So let's get on with the, let's get on with the programming.
And so I will simply steal this.
I'll cut and paste it.
And that will be that.
So I'm going to cut and paste this into my TextWrangler.
Now it's always dangerous cutting and pasting.
But I'm cutting and pasting from a slide which seems
to work a little better than cutting and pasting from PDFs.
And you can even get away with cutting and
pasting with PDFs as long as you don't freak out
when the first thing you see is a bunch
of crazy syntax errors because of characters being coded improperly.
So now I'm in this file, and I'm going to save this.
Save As, and I'll put this on my Desktop
in the lists folder.
And we'll call this day of the week .py in the lists folder.
Okay, and of course it's, syntax
colored it, which makes my TextWrangler
syntax colored it, and so I've got sitting
in this lists folder, I've got the day of the week .py.
Now one of the things that we've got to do is we've actually got to get our data file.
See this mbox-short.txt?
So the mbox-short.txt is sitting here on pythonlearn.com.
I'll go to the book page and you'll see this, all the stuff and code samples.
Including maybe a worked version of this example, you just never know.
[NOISE] Ii doesn't look like this one's here but
the file I'm looking for is mbox dot short dot t-x-t.
So there it is.
It's a mailing thing and here's one of those lines
we're looking at and we're going to there's many of them.
This happens to be the first line of the file.
So I'm going to make sure that I save this into the folder.
Go back to my Desktop and put it in lists and then save it.
And so.
At that point I should be able to go back and see ah-hah, I have mbox-short.txt.
Now the interesting thing is if you read this file, [SOUND] if you open this file,
TextWrangler's perfectly happy.
So we see from.
So if I was to do sort of a search, and I want to search for the string
capital from space. Okay?
And then I'm going to search next.
Oh, theres lots of them.
I'm going to make them case sensitive, okay?
So it'll only be the froms that are case sensitive.
There we go, and so in a sense our program is doing this little from where it's looking.
It's throwing away all the lines except those that start with
from and a space, because of the way the split works.
And then it's going to pull these, these days of the week, these Fridays.
So I'm kind of doing, if I was doing it by hand it
would be like find the line that starts with from blank, and then
grab this little text, and that's what I'm interested in.
Maybe I'm curious as to whether or not these
folks work on the weekends or on the weekdays.
And that's the purpose of this.
Okay?
So I'll close this.
Just, it's, that's what, that's the, Close Document.
There we go.
So here's my little program.
And so, let's, I can get rid of this now.
So now I'm going to go into my Desktop.
I'm in my home directory.
cd Desktop, cd lists. If I do ls, there I am.
I've got this day, day of the week Python program, and mbox-short.txt sitting here.
So I'm going to run it [SOUND] and there it works.
I'll get, I mean, well, works for some value of work.
So we get the same traceback the lecture slide suggested
we would, and thankfully that probably means the lecture slide's right.
So, so here we go.
We got, we got this one thing where the day of the
week is kind of coming out and then it dies on line five.
And it even gives us the line it's dying on.
So, it's complaining about list index out of range, and you
might be able to stare at this and maybe you're smart.
You're, you have enough skill already that
you're seeing these kinds of errors and you
just read it right away, know what the problem is, but that's not so much fun.
So here I am in line five, right?
I'm going to go right to line five.
And, and so the first thing I want to do on line five,
is I often add a print statement right before the line that's dying.
And I'll just print out something [SOUND].
Something random.
Just, it, you know, sometimes I print the letter a out, right?
And so now I'll save this.
And now I'm going to run it again,
and this tells me, just by putting a print statement in, what's going on here.
It's like, okay wait a sec.
This is the good line.
This is the line, I mean, this is the line I'm interested in.
As a matter of fact, I'm going to make another change
just to help me sort of visually see what's going on.
[SOUND] I'm going to put like a bunch of equal signs at
the beginning of this line and now I'm going to run it again.
So if I look, it's like, oh, dude, that's the good line.
That's the line I'm liking.
And here's my debugging, Something random, Something random, Something random.
So a lot of stuff is going on here.
And then finally it dies, right?
So finally, it dies here.
Now, it's line six because I added a line.
So the, so perhaps instead of printing
out Something random, I'll print something useful.
So, the first thing to do is to look at this statement, the one that's dying.
And say, what is the most suspicious thing in here, okay?
And, and so, if something's going wrong with this
words sub zero, it's saying index out of range, right?
Something going wrong with words sub zero.
So what's the deal?
So I'm not going to print out words sub zero, I'm going to print out words.
I'm going to say, like, what is in this words list at this point?
So now let me save that.
So instead of printing something random, we'll see a bunch of words go by.
And so here's that first line broken
into words, where from Stephen Marquard, Saturday.
And here's our good line.
And then here's one that doesn't have from as the first word, so we skip it.
Here's one that doesn't.
So let's continue onto where it's, you know,
we see lots of lines, everything's cruising along.
We actually see one that says from colon, but that's not what we're looking for.
So that one gets skipped too.
So make sure you skip that.
Now, here we are.
Here's the words.
Here's that list of words that we've split.
Now, this was the line before, here's the next line.
It's like, okay, what's going on here?
That looks like an empty list.
Okay, so I'm a little more curious, I'm going to,
I'm going to print, I'm going to print line out too.
[SOUND] And I'm going to put, just so I know what's
going on, I'll stick some, a couple pluses in front of that.
So the line will have pluses on it, and it'll print the word.
So let's run this again.
So I'm adding junk to this, right?
So now when I look at my things, the pluses mean that's the line I'm seeing.
Right?
I just put those pluses in for my own
visible, so I can visually see it more naturally.
That's the line and you can see how nicely it breaks
it into words, so it breaks this line at the space.
Oops, sorry.
That was not so good.
It, it breaks this line, spam probability line, at this space and gives me two words.
Right?
So here we are.
Two, here we go.
Here is this space, and then it breaks it into one word, and another word.
And so that's working.
But if I look here, that's the line that I'm on.
Oh, well that's a blank line.
Hang on a sec, hang on a sec. So let's
just real quick take a look at that file again.
Let's look at that file, and so here we go, here we go
So that's cool, that's cool, that's cool, let's look at the line.
And here we go, so if you look.
The last thing that we read successfully was this line right here.
This line is a blank line, so our program is choking on blank lines.
It's not failing on lines that work right, and it's
not failing on lines that don't have from in them.
It can handle the front line fine, it can handle the, any other line fine.
But it freaks out on the simplest of things, blank lines.
So the question is, how do we fix this?
[SOUND] Right?
How do we deal with this fact that this bit
of code, right here, is dying whenever there's a blank line, okay?
So, the way we do this is what's called the guardian pattern.
[SOUND] I'm going to get rid of this.
So one of the guardian patterns is it looks like this.
So, we don't want to get to this.
So we want to put something in front of it to
make sure we never hit it in a dangerous situation.
So it's if hmm, hmm, hmm, something, continue.
And this is if, we'll call this thing the safety check. [SOUND]
Check.
If some safety check, continue.
So that means that, you know, you know, if the safety check matches,
we're going to not fall through and not do this dangerous line.
So the question is what would we put in as the
safety check here to protect. This line is guarding this line.
That's what the guardian pattern means, is do
something before the scary thing that hurts you.
Okay?
So, one thing we can do is we can say, okay, well, what's words?
Well words is an empty list.
You know what?
If, if I got an empty list of words, I have no interest in this.
So if I say words is equal to an empty list, continue.
So what this basically says is.
You know.
Read the line, strip it, split it into words.
We'll print it.
If it's an empty list, continue and go up to the next line.
And then, if it is a non-empty list, it continues here.
And checks to see if the first word is From.
And if it is not, it continues, and then if it works, it works.
So this here, this line here, is the guardian line, it's
guarding the, this other line and you gotta do it in order.
If you do the guard, you have to
have the guardian happen before the line in question.
So now let's run it.
Oops.
Let's run it, and now it works great.
It's, we got a little too much crap, so we better get rid of this print statement.
It didn't give me a traceback, right?
So now I'm going to get rid of that print statement and look at that.
I'm getting the line Saturday, Friday, Friday, Friday, Friday, Thursday.
So it doesn't look like these people work weekends.
But it does look like they like finishing up stuff toward the end of the week.
So, there we go.
I mean, it's a little bit of, it's not much data to conclude anything solidly.
Maybe if we look at the bigger mailbox file, we can find something else.
Okay, so now it works.
Let me show you a couple of other things
that you could have used as a guardian pattern.
So we could say if the words we got back is an empty list.
I could say if the number of elements in words is less than one, continue.
That basically says, look, if the words that I got, if I got fewer than
one word, then I'm not going to look
for the first word, because that's quite dangerous.
So if I run that, it again works.
Another more direct, even potentially more direct, would be if let's move this up.
I could say if line is equal to empty string, continue.
So now what it says is, you know, if I'm getting a blank line,
I'm not even going to bother splitting it
because I know exactly what's going to happen.
So let's see if this works.
[SOUND] So this one works as well.
So now I've got, you know, a couple of codes, a couple of code paths through here.
Right? If, if the blank line, I do it after the strip to make sure it is really empty.
I skip blank lines the way, I come all the way down
and split and then skip lines that don't start with from this way.
And the only way I make it all the way down to here is if it is
a non-blank line and if the first word is a From, and then I do my thing.
So that basically is the notion of
the guardian pattern, and the pattern is simply,
if there is some code that might have a problem depending on perhaps user input,
put something in before it
that makes sure that you never get to the dangerous code.
And don't use a try/except for this.
That just would be tacky, tacky, tacky, tacky.
So that's the end of this this presentation.
So thanks a lot, and see you on the net.