Random Stuff

A sudden burst of black smoke mushroomed up into the air just as I pulled out of the grocery store parking lot. I picked up the phone and called 911 and headed in the direction of the smoke. It was only about four blocks from where I lived. The emergency operator let me know it had already been called in just as the police passed me by.

By the time I arrived, the police had already taped off the area. Fire engines were followed shortly behind. A duplex was burning quite rapidly and people from the neighborhood were gathering. I find it rather odd that the police were protecting the scene armed with shotguns, but regardless, I grabbed my camera, found a spot around the corner and started taking pictures.

I managed to talk to the occupant of the end of the duplex that was not burning. A mentally ill woman lived in the part that was burned up. As I stood there taking more pictures, I noticed the fireman getting a backboard from the paramedics on the scene. They weren’t in a hurry. That’s when I realized they weren’t rescuing someone but retrieving what was left.

I deleted the pictures and walked away.

* thought I would mention that meth labs are not common in around here.

From what I saw of the opening performance, it looked magnificent! Though, I could really have done with out the sports announcers giving constant commentary on it. I could also have done without the commercial interruption. From looking at the pictures of it on-line it looks like there was a good bit missed because even though it was recorded earlier they cut the commercials in just like it was live.

Also I have have issue with some of the crowd booing the Iranian team. These are world class atheletes, not the politicians who are causing all of the problems. I have a great deal of respect not for the leaders but the atheletes who have put all of their life into what amounts to a single performance in front of the whole world.

Amidst war, terror, fundalmentalism, and political pressure.

How many people can say they have risked so much for a single chance on a single day to shine?

Thought I would say hello to all of my former coworkers who over the last couple of days have let me know they read my blog.

Happy 08-08-08.

If you pay attention to the tech industry, or more specifically the Search Business, you have seen a lot of hype surrounding both Google and Cuil. Google recently announced they had reached the 1 Trillion mark. Cuil just came in with a bang saying they have indexed 120 Billion web page stating they have indexed more web pages than anyone else. So who is the biggest? Is someone lying?

Neither one is. It’s plain and simple marking. Both are playing with numbers and slight of terminology so that the detail is easily missed.

It is quite possible that Cuil has spent more resources on their crawler to scan web pages. They simply have managed to put a lot more effort into the gathering of data. It is quite possible they have managed to index 120 billion pages. This is no small matter. In my own exploration of indexing web pages, I found that the index really is a gigantic set of data. For 7000 indexed web pages, my index has 12 million records. It’s not optimized in any way, but you get the idea. Put in that context you can see that number may not be representive of what is important to the person doing the searching.

Look closely at the Google statement though. It says “1 trillion URLS”. It does not say they have indexed 1 trillion web pages. Those are two different things. Again, back to my own exploration of search engines. For 7000 indexed web pages, I have 300,000 URLS. Again, what google has done is no small matter, but needs to be put into context. What good are those 293,000 URLS if you can’t find them in the search engine.

So who is right? Which is better? What’s the meaning of all this? Pretty much nothing. It’s all marketing.

My personal opinion is that Google does a much better job of finding relavent data. Period. I do an ego search on Google, I find a few book reviews I’ve done on Amazon. If do the same on Cuil, I get the Amazon results but they are burried under a pile of websites that have hijacked Amazons book reviews. The reason for this could be that Cuil’s index is bigger, or that Google is doing better ranking of the same data. I can’t tell.

I do like the context and suggesting that Cuil does. And it has a slick interface. I also like the simplicity of Google seach.  I think the results that Cuil presents will begin to improve as they gain experience and figure out what is really important to the users. Google will then really have something to watch out for.

Headline today that came through one of my RSS feeds. I have to ask… Is this really news?

“Exercise helps keep weight off.”

A job has been handed to me on a silver platter. A job back at Microsoft. But this is not just any job. This is back to the same group on the same team that I worked with 12 years ago. Add to that all of the people are good friends. They are also people I work with exceptionally well as part of a team. In short it is the dream team I would have put together given a choice.

Interviews were a formality. Sallary negotiations consisted of them giving me what I told them I would take and then they added some more. But the real reason I accepted is the team I will be working with. I am being brought in as a senior member in the team. But make no mistake, these people are stars. People who I not only respect personally but professionally as well.

It was a choice that I did not take lightly. It kept me up at night thinking about this. Should I? Shouldn’t I? But, again, it all boiled down to the people.

Nope.

Ok, time for some boring stuff.

I’ve made some major improvements on the back end of my search engine. Response times so far are down to under a second for most queries. There are still some edge cases that still hang things up. I was able to make a query that took 15 minutes. But I think the most common cases are handled. Boolean searches are now possible. Scoring is a bit more accurate I hope.

I wiped the database clean and started repopulating it based on a different seed address (my own) so those that I have links to and those I have mentioned here will most likely be in there.

Right now about 5000 pages are indexed. 221,000 urls are left to be scanned. I read somewhere you should get about 8 links per page scanned, I am getting closer to 50 per page. I think that’s because I’ve scanned blogs and have managed to start indexing the BBC news site (I have a couple BBC links in my blog).

I can now run multiple crawlers across multiple machines across the internet if I so desire. I’ve limited them to 1000 sites per night and I am only running two, so I don’t piss off my gracious host (When your t1 is free, you try to be nice so you can keep it free).

Some things I plan on doing when I get the time…
1. Clean up the html
2. Create a logo
3. Write the typical “About”, “Legal”, and “Contact” pages
4. Create a page that lets people submit addresses to be indexed
5. Figure out a way to “update” the index for pages already scanned and how often to scan them
6. Write a better alogorithm for getting the abstract from the article.
7. Use the document title in the link instead of the link itself
8. Compress similar pages together (like subpages of a blog)
9. Write relevance ranking algorithm to compliment in text scoring.
10. Do more html parsing for relevance.
11. Figure out how to span multiple machines with the DB itself as it grows
12. Check out load balancing for both apache and the db. I’m sure my poor little web server wouldn’t handle a heavy load.

Maybe I should give my search site it’s own blog.
I think I might need to study up on information retrieval as well.

I’ve figured out some complicated items on search queries and how to handle them using forward indexes. I don’t have them implemented on the main page yet, but experimenting around I am now able to handle single word searches, multiple word searches and search phrases using the forward indexing.

There is still a long way to go yet. Multiple phrases and boolean expressions are not yet handled. Yet at the same time the queries now take 1/10 the time they used to.

http://rwack.com  (It also wouldn’t hurt if an ad or two was clicked )

There are also some things I need to implement which require I start over on the database so I get to wipe it out and start over.

Saying hello and introducing myself to someone from overseas in their native language.
But it’s so much fun!
Not that my ego needs to get any bigger…

This has to be one of my favorite interviews ever! Kudos to the guy being “interviewed”. The TV journalist is asking insulting questions to those waiting in line for an iPhone. He really picked the wrong guy (or right guy in my opinion) to ask “Have you ever seen a woman naked?”

My favorite part starts at the 1:34 mark where the interviewee clearly states “This is journalism to you?!” which strikes me as a pretty big insult to any journalist.

The price of gas is certainly a hot topic these days. I know personally it has hit my pocket book quite hard. A couple years ago I bought a car I never thought I would buy simply because it was getting too expensive to drive one of my gas hogs every day to work. When I did this I went from no car payment and $1200 a month for gas, down to a $575 per month car payment and a fuel bill of $300 per month. A decent amount of savings. Now that gas prices have doubled since then, I am almost back at the point I started at. For the first time since I was 20 years old, I am seriously considering riding the bus because it is getting too expensive for me to drive.

But I’m not angry about it. Far from it, really.

Over the last few years I’ve been in many discussions about fuel prices, fuel efficiency, electric cars, traffic, and a few other related topics. Usually there are a few different catalysts to the conversation or debate, but my argument as always stood pretty much the same. I usually emphasize two of my opinions. So far they seem to be playing out as expected.

People won’t change unless they are forced to do so. This is common for a lot of systems outside of people. A system does not change unless it gets stressed to the point of being either uncomfortable or even failure. People will not give up their cars unless it becomes too much of a burden to keep it. The environment, global warming, pollution, none of these have an impact on the driving habits of most people. Yes there are a few who take these things seriously, but the majority of the population will take no action regarding these impacts of a car. Why? Because they can still afford to drive.

Stress to the system will also drive innovation. The electric car has been around since the time of Henry Ford. But in the last 100 years, there has been a negligible amount of research to make electric cars go further, faster, and more reliably. Yet look at the gains in automotive technology. The worlds fastest car in 1908 could only go 141 miles per hour. It had only 135 horsepower. My “economy” car has twice the power, and with very little modification could go just as fast. But the electric car, up until recently has not really improved in performance at nearly the same pace. The power may be close, but the range was limited to about 50 miles at best. But recently, there has been real change in this area. Take a look at the Chevrolet Volt or the Tesla Roadster. Real change is coming. Why? Because people has started to demand it as a result of Gas prices. It is getting too expensive to put fuel in your car. Research is now actively happening. Making electric cars that cost less to drive and are just as convenient as gas powered cars. And they look good too.

But what is the price per gallon which people start serious making the change? The point at which it goes from only people who are early adopters to mainstream? I have always argued it will be when gas here in the U.S. goes above $5.00 per gallon. I’ve mentioned this over the course of several years as I talked to people about this subject. Recently when Honda came out with their hybrid car, and again when Toyota came out with theirs, and even yesterday when talking about the Tesla roadster. Daily we seem to get closer to that point. The new technology started to appear around $3.50 per gallon and really started to look like a serious alternative when the price went to $4.00 per gallon. Now, as we get closer to $5.00 it seems more and more like an electric car will be a reality.

I don’t know for sure. Those are just my opinions. But I think we are ready for this now.

pic 334

Originally uploaded by jesse_hires

Just another example of why things are so wonderful when the sun comes out here in Seattle

pic 419

Originally uploaded by jesse_hires

I always get a kick out of the July 4th fireworks on Lake union. This year was no different. I got what I think are some pretty spectacular pictures this time around.

I took about 400 pictures this time around of people, boats, and of course the fireworks display. Head on over to my flickr page to check them all out

Debbie and I spotted a Google StreetView camera car in my neighborhood this weekend. I wonder what the pictures will look like.

Well, OK. It’s not an accident. I decided to experiment around with writing a search engine. I’ve run into a few problems and still have quite parts I need to implement, but all in all, it has worked a lot better than I had planned.

http://rwack.com

There are the usual problems when you start looking at something big like this. Horsepower of the machines you are using for this, or in this case, machine. Memory is needed because databases are memory intensive. Drive space is needed because you are storing so much data to make this happen.

I did try to make it scalable. It is set up so that I can start adding more machines for doing different tasks as well as load balancing coming in from users doing searches. A real test of this, though, won’t happen unless I add more machines. That won’t happen unless I magically start making a lot of money off the Google ads running on the side.

There is a lot of work going forward. I’ve got some ideas about ranking web page relevance that I’m going to play around with. I need to work on the crawler and how it scans and stores links, also, this one part isn’t quite working how I want it to when I run multiple copies of it.

Another problem I have, which is mostly related to resources, is that I keep running into the big players on the internet. When I run into a site like Microsoft or Ebay, the crawler winds up spending all of it’s time indexing the thousands of links these guys have to their own sites. This wouldn’t be a problem with unlimited resources, but I want a more diverse collection of pages. And coming up with an exclude list is not a guarantee the results will be different. Imagine trying to come up with a list of all of the large sites on the internet?

So there it is. Search on a very tight budget.

Can’t talk about it, but had to at least say it. I’m excited.

Working for a start up company is all about Risk vs. Reward. Typically working for a start up means that there is a fair chance you will lose your job when the company folds. Or perhaps the company has too high of a burn rate and has to lay off a people because that last round of funding didn’t come in when it was expected, or perhaps that contract that would have made the company a couple of giant bags of cash was given to someone else.

There is also the chance that a start up company you work for makes it big time. You see the product you work on spread and become the center of the known world, or even a part of it.  You become part of something big. Hopefully all those stock options you got when you signed on were all worth the effort.

It is because of the above points you need to look at how you view your employment. Is it a job or an investment? Anyone can work for a company and do 9 to 5 work. But what is the payoff? Even at big companies, you won’t get far just putting in your time. Those that make the big pay checks are the ones who go the extra mile, take risks, and put in the extra time.

I look at who I am working for as an investment. Because of this, I look at the amount of risk I am taking versus the amount of potential reward. Will the payoff be big enough to justify the sacrafice of time, the stress of knowing I may be sent back home when I show up one day because the company ran out of money? How much sacrafice does my employer expect me to make and how much is he going to reward me?

Payday is one thing.  I can collect a paycheck anywhere. Does my employer expect me to make extra sacrafice for just a pitance and be happy with the knowlege I have a job? If the company sold tomorrow, what would the return be for the founders? Would the return on my investment match the amount of work I put in?

These are all things I have on my mind at the moment. When I started working for my current employer I made it clear that I viewed this job as an investment. Right now that investment is weighed.

I’ve been interested in search engines since I was first able to get results out of the WebCrawler. When looking at just what the web crawler did, I thought “I could probably do that some day!” but lack of time, resources, and most of all knowledge, I never did. I’ve kept an active interest in search engines, but getting real info was hard to come by. WebCrawler no longer does indexing, rather they aggregate information from other search engines now. But it started the search revolution as a project from University of Washington.

It’s at this point I really need to learn more. I need to go back to school. Why? Well, for the most part, I am self taught. The only formal education I’ve had in computer progamming were all in introductory classes about stuff I was already quite skilled at and using on a daily basis. I need hand holding. I do know some SQL (a language used for accessing databases), but anything outside a simple query is right now beyond my reach without more understanding of what lies underneath.

But back to the topic at hand. Off and on over the last few years, I’ve been working on my own crawler to index web pages and follow links. While toying around with this tonight, I stumbled on the original paper written by the founders of Google when they were at Stanford. When I started reading it I was fully expecting to be confused by all of the complex information presented. In actuality, it is clean, simple, and thorough much like the Google search interface itself.

So I sit here looking at the simple genius in that paper and wonder if I can come up with anything like it. I don’t know that I can, but it is still fun to toy around and solve problems for myself as a means of satisfying my curiosity.

What do you wish a web search engine would do?

Something my employer has done is on my mind. I’m irritated. Failure to follow through on a statement that was made. Not a broken promise, but close enough to break trust.

People there are still great to work with but I have lost respect for those in charge.

Comment from Debbie to Jake “You’re going to be just like your father aren’t you?”

Jake replies “I don’t see how that’s a bad thing.”

This is an experiment in using mencoder and ffmpeg to create a time lapse video.

I simply put my camera on my tripod, hooked my remote up, set the camera to auto fire. All other settings were on automatic. It came out to 287 frames between the two views in the video.

I then resized the images on my linux box using the imagemagick “convert -resize” tool to 640×480.

I then used mencoder (which in turn uses ffmeg) to convert the jpegs into this short clip.

I would like to make a couple more but I need to figure out how to improve the quality. There are a lot of compression artifacts in the video that are not in the individual pictures.

Today I put down new carpet inside the car, put the back seat back in, put in one of the front seats.

I also bolted the transmission to the bellhousing, spend some time figuring out what clutch fork to use and how it is mounted. Tried to locate some dowel pins for the clutch, but no luck there.

And now that dinner is settling, I’m headed back out to work on it some more.

Part of the reason I’m doing this is gas prices. Running around using up gas is not in the picture. It just costs too much, so much so, that I am going to start riding the bus. So rather than sitting around bored, I decided to start putting the car back together.

I need to buy a clutch equalizer bar pivot and locate those dowel pins before I can go further on the transmission. So now on to the interior while I can’t work on that.

I just got off the phone with the IRS. After several levels of voice menus, and then a hold time of about a half hour, I got to talk to a very pleasant lady. It turns out when they reviewed my tax forms, they missed the little box that says the income and tax from all of my securities sales is already reflected on my w2.

Since the notice also included an overpayment of Social Security tax, I now get a check from them for almost $400. That should make up for the lack of sleep this caused last night.

All others pay cash.

I got a bill yesterday from the IRS saying I misfiled my 2006 taxes and owe them $3800 by July 2nd.  Actually it says I owe them $3000 + $835 in interest (and they say loan sharks are crooks).

I don’t think I misfiled. They are adding my stock awards to my income, which is already stated on my w2 as part of my income and appropriate taxes withheld, effectivly saying I made $15,000 more than I really did. It’s happened before, and I can see where the error is.

Problem is. I am in a position of having to pay someone to straighten to prove I don’t owe them more money. But that won’t happen before the bill is due. So I have to pay up front, then pay some more to get it back. Losing situation all the way around.

It’s a sad state of affairs when you have to pay someone to figure out how to file your taxes correctly, or pay someone later to prove you filed your taxes correctly.

My sources of income aren’t that complicated either.
$$ Salary
$$ Stock Grant Sale (already declared on w2 as income, 33% withheld)
$$ Stock Purchase Sale (on company discount, already declared on w2 as income, 33% withheld)
Deduction for mortgage
Deduction for Kids
Deduction for Car Tabs
Deduction for Medical bills
and in WA, deduction for sales tax.

That’s it.

I just noticed that several letters on my keyboard have worn off so if you didn’t know what letter the key was supposed to be, you’d have a hard time with it. Also, all the keys on my keyboard used to have a matte like texture to them. A good majority of them are now smooth and shiny. You can tell where my fingers land on the keys, like I tend to use my left thumb more than my right when hitting the space bar, so it’s only shiny on one end.

This keyboard is only a year old.

Gas Prices are killing my budget. $75 for 16 gallons this morning. That covers about 4 days. For the first time I’m going to be riding my motorcycle out of necessity instead of pleasure. Time to buy a rain suit.

I wound up leaving a party last night a bit early. It was for a friend who is shipping out to Iraq in a week. Debbie wasn’t with me, but my friend’s parties are usually quite safe. Attractive single women have not been commonplace at any of his parties in the past.

There was an open pit fire going in the back yard where a couple of people were keeping it well stoked, so it was a nice place to hang out while others from the party filtered by and chatted. It was also a good bit quieter than inside the house making it a good place for conversation. I got to meet and talk to lots of different people as the night went on.

Joon and her friends showed up, my friend introduced her to those of us sitting around the fire.

“Kobanwa.” she said with a slight nod to the head.
“Kobanwa.” I responded back in the same manner. Kobanwa is a Japanese way of saying Good Evening.
“Ah! You speak Japanese!” she said in Japanese
“A little, but not very well.” I said back in Japanese. Then I followed up in English that I didn’t know any more than that. Next thing I know, she is sitting on my left and we are talking about Japan and Okinawa.

Soon after Tina showed up. She was a neighbor of my friend from just up the street. A couple others from the party were openly trying to pick her up and failing miserably. It wasn’t long before she had that “Get me out of here” look on her face. I was watching this when she looked my way. I couldn’t help but smile, an acknowledgment that I knew what that look was. But then she smiled back. I actually turned my head and thought “Shit, I shouldn’t have done that.” Next thing I know she is sitting on my right.

The conversation was enjoyable, but they started to get a bit obvious. The hand on my arm when making a comment. The slap on the shoulder when I made a joke. The scooting closer and closer on the bench. I think the most interesting was I now had an entourage of two when I went up to get myself a drink. It was flattering. But as they both became more obvious as well as competitive for my attention, I decided to call my exit.

That’s the first time I’ve actually been uncomfortable with the attention I was getting.Flattering yet, but I was just looking for conversation. Had I been single I most definitely would have stayed and I suspect it would have been quite interesting to say the least. But I am not single.

I called my girlfriend as soon as I was back in my car and on the way home.

It’s a joke with my girlfriend that I can’t help it. Just being me is what “it” is and I can’t turn that off.

My son had mentioned his girlfriend a few times in passing conversation. From what I understand they have been dating for a couple of months, which seems to me an eternity for a 14 year old kid. He hasn’t revealed any information about her except when pressed by me. Questions like “Is she smart? Is she cute? What’s she like?” and so on. The usual curious father type questions.

I met her for the first time today. She was helping my son with his homework when I got home today. She is very smart, very cute, polite, very well spoken. The type of girl any father would be glad to see his son dating.  I took her and my son to give her a ride home. I was saddened by the fact that she was concerned about being seen because of very racist people in her family that wouldn’t understand why she was dating a white guy.

I kept my mouth shut which is contrary to my nature. But any kid who has to put up with that from their own family and has the courage to step above it has my utmost respect. A very bright girl indeed.

Just for fun I updated the engine and some of the website that I have for groveling car pictures off the internet. The engine code is much cleaner, the database is smaller, and searches are faster.  It’s also got a separate component for loading the pictures so the image information is preserved.

Check it out: carpicarchive.com