blog of the software engineering student and occasional tweeter
Posted on October 13th, 2014 by Milos Ivanovic Filed under Google, Networking

It really isn’t every day that your Google Search request fails with:

Server Error
We're sorry but it appears that there has been an internal server error while processing your request. Our engineers have been notified and are working to resolve the issue.
Please try again later.

My two subsequent attempts also failed with the same error, but it got back into working order after that.

I quickly searched this message on Google Search – after it was working of course – and found something else you also don’t quite see every day:

38sec

Crawled “38 secs ago”? Someone else’s discovery is definitely related.

Just minutes before however, 8.8.8.8 and 8.8.4.4 could not be reached from my location, which are usually reached in around 24ms. Over 30 minutes later, it was finally rerouted to somewhere else with a latency of 174ms. I’m not sure if these issues are related, but something has definitely gone wrong somewhere.

Google_last_10800

A 30-minute outage and then a reroute to a location outside of Australia.

I wouldn’t be surprised if some admins at Google were running around with their hair on fire.


Posted on June 18th, 2013 by Milos Ivanovic Filed under Hardware

I decided to treat myself to a new computer. What I’m showing below is just a tiny part of it. Which means, as a whole it’s going to be one massive beast of a computer…

NVIDIA GeForce GTX 770

NVIDIA GeForce GTX 770


Posted on May 21st, 2013 by Milos Ivanovic Filed under Google, Security, Software

I was just watching, as part of YouTube Comedy Week, a live broadcast of Day 1 — YouTube FanFest with HP, hosted in Singapore. It featured a Google Hangout with one of my favourite YouTube celebrities, Natalie Tran aka communitychannel, who lives in Australia and was unable to make it to Singapore for the live event. Now, you would expect IT security to be decent when it comes to a YouTube event like this one, even more so considering the event was sponsored by a large corporate company like HP. But it really wasn’t, and I feel both ends are to blame.

The Google Hangout session was screencasted by the host at the event in full-screen view, and by that I mean you could see the entire screen real-estate, taskbar, notification area and menu bars included.

google-hangout-communitychannel

You were therefore able to see the Google Hangout session identifier very clearly in the URL, in 1080p HD. Take a look above; it’s https://plus.google.com/hangouts/_/0b03ee704e9d7337b73bf36109b1168aab667b76?hl=en. That means any one of the 5,300+ people watching live on YouTube were able to simply type the URL into their browser, reading it off the YouTube video like I did, to hijack the Google Hangout session and talk to Natalie Tran (♥). And two people did at 2 hours and 37 mintes in. One of them apparently flashed a naked photo, which happened while the camera was pointing at the audience and you could see everyone laughing. Neither the host nor Natalie Tran knew what was going on but they went along with it for a short while, with Natalie awkwardly joking about how they are her Internet boyfriends and such.

While that is funny, it’s not acceptable. First of all, the browser used at the live event, which as seen above was Google Chrome, should have been in full-screen mode obscuring any URLs from view. Secondly, and this is the real issue: Google developers should have been less naïve and realised that just because it’s not probable that an outsider will be able to view the URL of a Google Hangout, it doesn’t mean it’s a good idea to show the session identifier in plain text in the URL. They should have used a session cookie, which is not so obvious and oblivious to those who can see the screen of a Hangout user with a Hangout in progress.

Knowing the session identifier, anyone is able to join a Google Hangout without authorisation from anyone presently in the Hangout. Since I don’t use Google Hangouts, I don’t know if there exists a way to force authorisation before joining, showing video and speaking on a Hangout, but if there isn’t, this is an absolute failure of product security by design and requires urgent attention.


Posted on January 30th, 2013 by Milos Ivanovic Filed under Gentoo, Hardware, Linux

It’s finally here! The replacement for my Raspberry Pi which has caused many issues with its poorly-written USB drivers!

ODROID-U2

ODROID-U2


Posted on June 24th, 2012 by Milos Ivanovic Filed under Hardware, Linux

So… I bought a Raspberry Pi. It was really tasty, so I had to buy 300 more. Ok, so it’s not actually raspberry-flavoured and it certainly has nothing to do with pies, or the mathematical constant π either.

What it is related to, however, is electronics. In a nutshell, it’s:

  • an ARM-based, fully-featured, embedded computer scaled down to the size of a credit card
  • it has a 700MHz stock-clocked ARM CPU (ARM1176JZF-S), and
    • 256MB RAM
    • an SD card slot to store the operating system
    • HDMI + composite out
    • 100mbit LAN
    • a GPU fast enough to play full HD 1080p video
    • 2 USB 2.0 ports
    • some low-level peripherals
  • and it costs just $35USD

What a neat little thing. You can read more about it and its initiative at the Raspberry Pi Foundation’s website.

Mine actually arrived about a month ago but I never found the time to sit down and blog about it – well, now I can! And I’ve got some interesting concepts to share.

Raspberry Pi with a GPS and 3G dongle attached

My Raspberry Pi, with a 3G modem and GPS module connected, all running off a 16Ah battery

With this setup I was able to make a live GPS tracker, using as well, of course, Google’s Google Maps API. I’ll show some screenshots of that soon.

This is one step forward in the project me and a few friends have been thinking about, which is to make a quadcopter (camera included) that is controlled over the Internet. We’re still very much in the planning phase, but we have definitely made some progress and I’ll be glad to keep blogging about that as it gets formed.


Posted on April 20th, 2012 by Milos Ivanovic Filed under Hardware

About a week ago, I was removing a slowly-failing Seagate ST31000340AS (nearing 5 years old) from my file server. I had backed up all the files onto a Seagate ST32000542AS so that I could return the failing drive to Seagate for a free replacement unit. Upon removing the drive and returning to boot the server again, I noticed the ST32000542AS was no longer identified as such but instead as ST_M13FQBL. In the OS, the drive showed up as having 3.86GB capacity and Windows was complaining that the drive was not initialised (nor could its contents be accessed in any way). With some quick Googling I realised that this can happen if the SATA cable becomes slightly loose – and that was the case. I reseated the cables and we were back in action.

However, the woes continued.

Since the former drive was no longer present (and it used to contain my OS), I had to reinstall Windows on another drive. I decided to give Windows 8 Server a go to compare differences from Windows Server 2008 R2 (why not?). It crashed halfway through the install and then complained about winload.exe being corrupt. I laughed and blamed Windows, until I finally realised it wasn’t actually Windows’ fault. In the BIOS, the drive was shown to have a capacity of 0MB with 0 sectors.

Ouch.

I downloaded the most recent copy of Hiren’s BootCD to save my arse (since I had no other OS to boot from to try and find out what the heck was going on), booted it via USB, and found that in both the included version of Mini Windows XP and Linux-based Parted Magic the drive was shown not to be responding in any way. I either got I/O errors or other generic failures when attempting to access it.

At this point I was more annoyed than distressed because I knew the drive was not experiencing any mechanical failures; it spun up, calibrated and sat idle waiting for tasks. I just couldn’t instruct it to do anything.

Much Googling proceeded that lead to many questions – is it a firmware issue? Is it my drive’s PCB? I found there was a firmware update available, from CC34 to CC35; perhaps that would fix it I thought. I updated it, but nope, same deal. 0MB and 0 sectors.

I felt like I hit a brick wall at this point, but I kept researching. I found a replacement PCB on eBay for $50 which I felt was a bit pricey (still cheaper than commercial recovery solutions) so I’d have to be sure it was the PCB before I bought it. The seller had some nice suggestions about contacting them if I had any questions, and that’s what I did, mentioning I already tried a firmware update to no avail. This was their reply:

Hi Milos,

Thank you very much for your message.

I don’t think a PCB will fix this problem, it is most likely a firmware issue. This part of the firmware is stored on the drive platters, not on the PCB. Please give this or this a try. It is supposed to fix the firmware problem, hopefully it helps! If you need help locating the necessary equipment, you can try a pre-made kit from our business partner.

I hope this helped and please let me know if you have any other questions.

 

Best Regards,

Kevin

PCB Solution – Hard Drive Circuit Board Specialists

What a nice guy – not only did he stop me from buying something that he knew wouldn’t help me, but even linked me to what in the end actually solved my issue.

On first glance the first link disappointed me because it said “These are instructions for fixing a Seagate 7200.11 hard drive that is stuck in the BSY state. This can be determined by the fact that it won’t be recognized by the computer’s BIOS. If your drive is detected, but shows up as a size of 0, then you’re drive has a different problem and these instructions will not apply.” Needless to say, I didn’t apply the fix as it told me that this clearly wasn’t for my drive, as mine did show up in the BIOS correctly but with a size of 0, so I put it aside and continued searching for a solution.

After about 48 sleepless hours I stumbled across this post, which shows somebody with the exact same model hard drive as me had the exact same issue I was having, and was able to resolve it using the 0 LBA fix mentioned here. By that point I was pretty sure the fix would work for me too, even though it’s stated all over the place that this is only for the 7200.11 series (my ST32000542AS is a 7200.12 series). I ordered myself a USB to RS232 TTL PL2303HX auto converter module off eBay instead of following the suggested procedures because this module did it all-in-one; I didn’t need multiple connectors or converters. To top it off, I didn’t even have to wait long since I just ended up borrowing a friend’s the next day!

The procedure

I plugged the USB module into my netbook, attached 3 wires to the correct serial port connectors on the hard drive, powered the drive the same way as always, and I was in business. I configured PuTTY (my terminal emulator of choice) with the parameters mentioned on the site I linked earlier, and opened a connection with COM12 which is whatever Windows decided to pick as the port when I inserted the module.

The following are the commands I entered, each line followed by a newline character (enter):

CTRL+Z
m0,2,2,0,0,0,0,22

Giving me this output:

User Partition Format Successful - Elapsed Time 0 mins 00 secs
Zone re-format was skipped.

Once I saw that output, and only once I saw that output, I powered down my drive, plugged the SATA cable back in and booted the machine, as instructed. And, just like that, it was fixed.

So, hearing it from another successful fixing of the 0 LBA problem with a 7200.12 series drive, don’t necessarily be taken aback if you have the same issue and you don’t have a 7200.11 model which everybody is claiming is the only model for which this fix is made for.

Good luck and remember, even though I’ve said what I’ve said, my comments come with ABSOLUTELY NO WARRANTY. If you bricked your drive following these instructions then I’m sorry, but it worked for me.


Posted on March 12th, 2012 by Milos Ivanovic Filed under Software, University

Those who have been following will be glad to know that I have completed the University of Auckland Exam Grade Poller, written in object-oriented Python. If you haven’t been following or don’t know what this is, well, if you happen to go to Auckland Uni this might benefit you if you would like to be notified exactly when your exam grades are released on Student Services Online.

Let’s jump straight into it – here’s an action shot of the program’s main menu.

It is currently a command-line application with a b0rked ASCII UoA logo.

Usage should be straight forward; just fill in what it asks for and you’re away. You can either manually check grades for a given semester (option 1) or use the beefy polling functionality to get informed of new grades for your current pending semester automatically (option 2).

When I wrote the alpha version of this program last year for the 2011 Second Semester session, I ran it as a daemon on one of my servers and had it post directly to the 1st-year Engineering Facebook group I was part of, effectively informing 500 students that new grades had become available on Student Services Online.

Behind the scenes I went a bit further and even had it deliver me a personalised text message containing my individual grades as they were released, using Clickatell’s HTTPS API for its global SMS gateway system. As a surprise I thought I might as well add the mobile numbers of my friends who were also studying Engineering and inform them of new grades via text message too (these messages didn’t contain their grades though, as this was outside of my control; it just had the same message as the one in the image above).

As for the current edition, in order to be notified you need to download and execute the program on your own machine and set it up to poll for new grades (option 2). Once it detects a disparity in your results, it will invoke 10 system beeps, giving you plenty of time to become nervous before it displays your shiny new grades on the screen in a neatly-formatted ASCII table (trust me it’s much neater than that logo). Granted this isn’t as cool as getting a text message, I can’t afford to have the program send everyone personalised text messages on behalf of my account. I would if I won lotto, though!

If this fancies you and you would like to use it, feel free to download the Windows or Mac executable (Mac OS X App planned) or if you have a phobia of executables thinking I might steal your university credentials or something ghastly like that then you can get the source for the API and the poller itself instead.

Download links below – version 2.0.1.8 – last updated 07/07/2014

 

Windows executable (version 2.0.1.8 – current)
uoaegp.exe (3.7MB)

Mac OS X executable (version 2.0.1.8 – current)
uoaegp (5.8MB)

Python source (version 2.0.1.8 – current)
ssoapi.py (10.8KB)
uoaegp.py (10.7KB)
both files are required

Lazy changelog

2.0.1.2 — 12/03/12
– initial public release after a plethora of tests

2.0.1.3 — 26/06/12
– fixed incorrect semester selection due to upstream behavioural change

2.0.1.4 — 30/06/12
– additional fixes for semester selection issues

2.0.1.5 — 05/07/12
– show date and time of disparity detection

2.0.1.6 — 27/11/12
– fix: a change on the UoA login page caused ssoapi to reject all credentials
– permanently enabled a workaround which fixed an issue in both manual and automatic semester selections that appeared around the fourth quarter of 2012

2.0.1.7 — 06/07/2014
– added important fail-safes
– improved exception management
– additional fixes to attempt automatic recovery from erroneous states

2.0.1.8 — 07/07/2014
– fixed inability to view individual semesters manually (option 1) –  regression from 2.0.1.7
– less overwhelming error announcements

Tips and to-be-FAQs

I want to use the Windows executable.
Great, go right ahead. If you have runtime issues, try using the source instead.
This is compatible with Windows only.

I want to use the Mac OS X executable.
This is not quite the same as the typical Mac OS X apps you may know, as it is a single binary file rather than a .app directory. Here are the instructions:

  1. Download the file and note which folder it was saved in (/Users/Milos/Downloads/ is probably default, but with your own username in the middle; that is, your Downloads folder).
  2. Search up Terminal in Spotlight (the top-right magnifying glass icon) and open it. You now need to give the file execute privileges so that you can simply double-click it in the future without having to manually open Terminal. This is done by using the chmod command, explained next.
  3. Type `chmod +x Downloads/uoaegp` without the backticks (`) and press enter. That’s it! You can now double-click the file to use the program.
  4. Optionally move it to another location, i.e. Application directory if you really want to. 

If all you see is a blank terminal, chances are it’s hiding behind that window.
This is currently definitely compatible with Mac OS X 10.6 (Snow Leopard) and higher but should theoretically work with Mac OS X  10.4 (Tiger) and higher.

I want to use the Python source code.
You are going to require at least Python 2.5 with the lxml module installed.
This is compatible with Windows, Mac OS X and Linux as long as Python and any dependencies are installed and able to be located by the interpreter.

My poller is picking the wrong semester!
You are most likely using an outdated version; please update to 2.0.1.4 to address this issue.
If you continue experiencing this with the latest version, let me know by leaving a comment below.
Thanks and sorry for any inconvenience.

I left it on overnight and when I checked it next it had an error (something about __getitem__)?
This is harmless no matter how many times you see it. As long as the timer is consistently counting down, the program will still work as intended.


Posted on March 6th, 2012 by Milos Ivanovic Filed under Hardware, Linux, Networking

I decided yesterday on a bit of a whim that I wanted to replace my current network router, which is not exactly your typical router. While you could say that typical routers consume something like 2W of energy, come in a decent-looking plastic enclosure and have lots of continuously-flashing lights, mine’s actually a full-height computer in a boring old tower box. What’s worse is that it has an Intel Pentium D 960 3.6GHz dual-core processor, which loves to eat at the power bill. Those who realise that a router is not meant to be a power-hungry device like this would probably be yelling at their screens right now, and it’s true; 130W TDP for just the CPU is far too excessive to be run 24/7 given that an average router CPU (let’s say 400MHz or so) will consume a Watt or less.

But why would you even use a full-size computer as your router? Well the size is definitely questionable, but it’s the fact that it’s a general computer that I’m trying to emphasise. When I say this I mean it’s a computer that you can install anything on, and tweak to your heart’s content (so definitely not an Apple Mac). For example, I installed Gentoo Linux on mine (some people might have jumped ship after reading that) giving me OS-level customisation as well as the ability to open the box and add components to the motherboard.

When you buy a router from the local store, granted it’s probably fine for most people’s homes as-is, you’re not really able to tweak the components to for example add more memory or insert an additional network card if you need it, and you’re definitely not able to tweak the OS without re-flashing the firmware with a third-party variant like DD-WRT, OpenWRT, Tomato or others which can still be limiting.

I don’t like having those restrictions.

I like something that I can tailor to suit the task exactly, which in my case means making it more generic to allow for future integration with other systems. And what do I mean by that? Well, handling HDTV network streaming will be one, holding a second fail-over 3G connection to switch to automatically in case the main connection goes down is another, and most-importantly having the ability to do absolutely anything from an unrestricted UNIX command-line prompt on an operating system of entirely my own choice. And that’s why I bought the Intel DN2800MT, to replace my previous router (which, in case you’re wondering, had an ASUS P5VDC-MX motherboard that had slots for both DDR and DDR2 memory – what the hell?)

The Intel DN2800MT Mini-ITX motherboard

This thing is great. It has an on-board Intel Atom 1.86GHz dual-core processor and- wait, that’s still overkill. Ah but before you say more, let me mention that it takes 12V DC input instead of the usual AC and only consumes about 6W of energy while in use. That’s a massive improvement over my current setup while allowing for much greater customisation.

If you haven’t already spotted, the best thing about this motherboard is that it’s of Mini-ITX form factor. That means of course that it’s very small compared to a typical computer motherboard (which is usually ATX and requires active cooling, although it’s also common to have mATX which really isn’t much bigger), giving me the chance to get a nice slim box for it.

And here it is.

M350 Universal Mini-ITX Enclosure

Okay so we’ve now entered the grey-area. That’s the case I’d like to get, but I know it isn’t going to have any space for a PCI-E x1 card, which I will need for the secondary NIC as the motherboard only has one built-in (sadly). I’ve been researching a potential solution and I’ve found that you can use right-angle riser cards to switch what once was the height to now go length-wise in the case.

PCI Express x1 right-angle riser card

That’s exactly the remedy I need for the expansion slot. It probably still can’t fit in that tiny case, though, but a slightly taller one is still better than a full-size Mini-ITX case, which is comparatively twice or thrice as high as the one already pictured.

More soon.


Posted on November 23rd, 2011 by Milos Ivanovic Filed under Gentoo, Linux

Okay so I got myself a Logitech C910 HD webcam (on eBay, brand new for $59USD) which has native support for the linux UVC driver, meaning it’ll work on any linux distribution if you compile it with v4l2 support. That was great and all, but it took me far too long to find out how to actually change the output resolution for the webcam to be 1920×1080. So, for anyone who is on the same boat, here is what I did.

As I use Gentoo, I had to install the media-tv/v4l-utils package (other distributions should have a similar name). Once installed I played around with the binary files and found v4l2-ctl which is used to configure the webcam once it has been plugged in (only lasts per USB session).

Running v4l2-ctl –help yielded a truckload of output which was unpleasant at first, but the solution was found nevertheless. The important parameter is:

-v, --set-fmt-video=width=<w>,height=<h>,pixelformat=<f>
                     set the video capture format [VIDIOC_S_FMT]
                     pixelformat is either the format index as reported by
                     --list-formats, or the fourcc value as a string

which will send a control to the webcam to tell it to output in the given format and resolution. For 1080p I used -vwidth=1920,height=1080,pixelformat=YUYV

Once set, it was all kittens. Open the video device (/dev/video0 for me) with your desired program and it will use the format and resolution set in the above step.

Do note that the webcam does not remember these settings so when unplugging and plugging in again, you’ll have to repeat this command.


Posted on May 20th, 2011 by Milos Ivanovic Filed under Personal

I went on a fantastic trip to Europe with my girlfriend a few months ago! Wow that shows just how much of an avid blogger I am, posting bleeding-edge updates about my life… D:

I really need to blog more often, and I’ve been told that a lot, too.

Nevertheless, here is a photo from the trip (it was in an uber fancy house, with servants and everything!)