Setting fake user agent in Polipo

I use Polipo as a caching web proxy for my VPN and for some simple anonymizing. It’s a great software.

One thing bugs me though: when censoring User-Agent, as intended, it does not send a User-Agent header. But: Sometimes poorly written web scripts break on a missing user agent string or web scripts take a missing user agent string as an indicator for a bot and block the request (Wikipedia does that for example).

I have created a tiny patch that sets “User-Agent: empty” header when censoring “User-Agent” so that web scripts do not break or block the request. There are only four new lines and you can set the fake user agent string in config.h:

diff -cB ./polipo/config.h ./polipo_custom/config.h
*** ./polipo/config.h 2011-07-17 12:48:42.000000000 +0200
--- ./polipo_custom/config.h 2011-07-17 13:22:06.000000000 +0200
***************
*** 35,40 ****
--- 35,41 ----
  #define CONFIG_INT_LIST 12
  #define CONFIG_ATOM_LIST 13
  #define CONFIG_ATOM_LIST_LOWER 14
+ #define USER_AGENT_STRING "empty"
  
  typedef struct _ConfigVariable {
      AtomPtr name;
diff -cB ./polipo/server.c ./polipo_custom/server.c
*** ./polipo/server.c 2011-07-17 12:48:42.000000000 +0200
--- ./polipo_custom/server.c 2011-07-17 13:25:28.000000000 +0200
***************
*** 1678,1683 ****
--- 1678,1686 ----
          goto fail;
  
      if(request->request && request->request->headers) {
+  if(strstr(request->request->headers->string, "User-Agent:") == NULL) {
+      n = snnprintf(connection->reqbuf, n, bufsize, "\r\nUser-Agent: %s", USER_AGENT_STRING);
+  }
          n = snnprint_n(connection->reqbuf, n, bufsize,
                         request->request->headers->string, 
                         request->request->headers->length);

You can check your user agent string with http://whatsmyuseragent.com/.

There is no PayPal for me

Every now and then colleagues or customers ask me whether I can do PayPal transactions with them.

Unfortunately, I can not. Let me explain:

Until some 7-8 years ago I had been a happy PayPal customer with a positive balance of x.y EUR. Then, from PayPal’s point of view, I made a “suspicious” transaction of exactly x.y EUR.

For PayPal, this was an attempt of fraud. I cannot remember who PayPal thought was the victim in this case but it does not matter either.

PayPal froze my account and demanded some proof or acknowledgement that the transaction was no fraud. They suggested sending them a utility invoice that should prove that I am real. Unless you do not know, in Germany it is absolutely uncommon that a company asks for a copy of a utility invoice. Maybe in some other less developed country this is considered appropriate, but I do not. This is odd and strange and none of their business.

I saw no need to prove my innocence, because… you certainly remember: In a constitutional state you are innocent until proven guilty.

My account remained frozen for three years until PayPal started inviting me back to use their services again.

Thanks, but no thanks.

Most companies pay good money for customer leads but this company seems to have a Customer Repellent Department.

So, there is a good reason for my PayPal abstinence. I hope you understand I can not risk my balance being frozen by a rampant company.

Outsourcing freelance jobs FAQ

I like outsourcing work to other freelancers when I am short on time myself. Over the time, I am often asked the same questions again and again. Obviously it is time for an “Outsourcing freelance jobs” FAQ. :)

Question: Should I start working even though I have not been told to?
Answer: Please don’t. Start working when you have won the bid and we have discussed the project’s outline.

Question: Should I set an hourly rate that is below 50% of my country’s average web development hourly rate?
Answer: This would be most certainly below your country’s average living wage. Please raise your hourly rate to a fair level that enables you to sustain life.

Question: Give me job!

Ok, this is not a question. But I have an answer for that, too:

Answer: While it is not necessary to speak and write perfect English (which I do not myself) I try to keep a certain level of politeness in communication, with appropriate salutation, “please” and “thank you”. It makes daily routine more bearable. :) If you keep it the same way, you raise your chances of being considered (and being considered again after the project is completed).

Question: I do not have any references. Is this a problem?

Answer: Yes it is. If you just have graduated school or university, make pro-bono projects and use them as references. By the way, that’s how I got my first job in IT business.

Question: I am located more than 2 time zones away from you. Is this a problem?

Answer: Yes it is. I prefer freelancers not too far away because I do not like to wait too long for emails. When I search for freelancers that can help me with a project most of the time it is quite urgent.

Question: I am a screen designer. But I can code too! Would you choose me?

Answer: That depends on your coding references :)

Question: I bade on your project and I am the most expensive, being way above average. Is this a problem?

Answer: Regardless of any other advantages you may have: yes it is. Nobody wants to pay more than he has to.

Question: I bade on your project and I am the least expensive, being way below average. Choose me now!

Answer: Regardless of any other advantages you may have: no I would not. Either you underestimate the scope of the project intentionally or unintentionally (that’s bad either way) or you are offering dumping prices. That’s bad too.

That’s it for now. This list is most likely incomplete. I will add more questions and answers over time.

Memory footprint of Facebook’s Like button iframes

I have made a rough review of Facebooks Like button memory footprint. Since every like button sits in an iframe I am assuming that this would have a heavy impact on a browsers memory footprint. Web developers should keep that in mind when putting many of them on a single page. Not to mention that each like button comes with it’s own Javascript and CSS files and that Facebook is most certainly tracking even people not using the button and so on and so forth…

There *are* very good reasons not to use Facebook’s Like button regardless of its memory footprint.

The tests were made with Firefox 3.6.10 on Vista with Firebug and some other plugins enabled on a client’s WordPress blog. I have recorded the amount of iframes, peak memory value (the highest value during or shortly after page load) and idle memory value (20-30s after page load).

Keep in mind that this is by no means scientific. YMMV.

Amount iframes ~MB peak ~MB idle
1 105 50
11 108 57
21 125 70
31 149 80
41 175 100

Obviously idle memory footprint scales at about 1,0-1,2 MB per iframe. That is quite a lot.

And do remember that each iframe comes with its

  • own HTML,
  • inline JavaScript,
  • inline CSS,
  • external JavaScript and
  • external CSS and
  • at least one image.

I have not counted the exact amount of requests though I expect ~4, maybe ~5 HTTP requests per Facebook Like button iframe. With 40 of them on one page that sums up to 160 HTTP requests just for the Facebook Like buttons.

From a Green Computing point of view it is quite insane.

My conclusion is if you *really* want to use Like buttons that you do not to put more than a dozen or half a dozen Like buttons on a page.

CSS Level 3 Media Queries

In my humble opinion, the upcoming CSS Level 3 Media Queries are one of the major improvements in web development.

Yesterday I have implemented those in my web site so that it adapts to various browser window widths.

See them in action on Youtube!

See them in action on Youtube!

Try it for yourself and tell me what you think :) (CSS 3 media queries are currently supported in FF 3.5+, Chrome, Opera, Safari and IE9+)