For a while, traffic went in but didn’t come out

The attack, if that’s what it truly was, involved looking for well known weaknesses in popular blogging and administrative tools that we don’t run. These sort of investigations occur almost routinely and usually only come from one IP address. The one on Wednesday seemed to involve a fairly large number of machines, all doing the probes, and doing them repetitively. The IP addresses came from two large, but otherwise anonymous, Unicom subnets, based in Beijing. They looked to be compromised PCs and I was forced to block web-access from the networks involved for a day. But only after I noticed things slowly grinding to an almost-halt.

Not only did network traffic get swamped but one of the Apache-based web servers did too, the one that provides the Examplar and the Blog. Some of this was down to poor error handling that I’d written, making things worse in an ever-so-slightly witty way. So mea-culpa too.

As the only traffic noted from this part of the world tends to be search engines and pointless weakness-probes, I’m sure that there wasn’t a great deal of inconvenience in firewalling Beijing for the day.

Anyhow, things back to normal now. The remedial stable-door bolting is in place. As are the crap metaphors.

Thanks once again to the readers for the feedback. I often write these posts thinking that the only things that read them are the odd passing, and now sometimes malicious, robot. It’s nice to know that there’s the odd human out there too.

Update: The outage is ‘planned maintenance’ for a couple of days in prime-shift, if a recent Tweet is correct by @EURESjob is correct. 2013-05-20

Update: EURES is back up, with no changes, as far as can be seen. 2013-05-21

• Jobs posted last week (Week Number 20): 19786
• Vacancies were still live this week: 18714 (95%)
• ‘Agency’ postings (from third-parties): 4042 (20%)
• ‘Job Warehouse’ postings: 1158 (6%)
• ‘Company Confidential’ postings: 5403 (27%)
• Appear to be ‘apprenticeships’: 505 (3%)
• Appear to involve ‘catalogue distribution’: 943 (5%)
• Part-time: 4865 (25%)
• Postings declared self-employed: 1879 (9%)

This is somewhat like earlier attempts found elsewhere on the Blog. It’s early days, for this examination of Universal Jobmatch, and it may be expanded in future weeks. In examining the data we’ll try not to be judgemental and leave conclusion-drawing and ranting to others. In spite of this your feedback is appreciated, as ever.

This post is automated, but there’s a Human of some description behind it. E-mail him to express your disappointment in this.

Gone

All these ‘expiry’ files have data in ‘Comma Separated Format’ with the following fields:

jobid

The Job ID, correlates with that found in other CSV files on the FTP site.

title

The Title of the Posting

location

The location, as originally recorded on Universal Jobmatch (UJM). Some of these can be a little odd, UJM doesn’t do geography that well.

added

The date that the posting was noted internally on Universal Jobmatch.

noted

The date the posting was noted by the Archive.

expired

The date that the posting was noted as being missing from Universal Jobmatch.

In addition this work has caused significant clean-up of the existing data, as re-examined data was re-parsed with more modern code that is now aware of the large amount of Microsoft Word generated embedded HTML in some of the more mechanical postings. In consequence the first Quarterly dump has been re-run and re-written UJM-historic-Q1-2013.csv.gz. This (and subsequent files) will now have an expiry date if the posting has been withdrawn from Universal Jobmatch too.

• Jobs posted last week (Week Number 19): 15306
• Vacancies were still live this week: 14513 (95%)
• ‘Agency’ postings (from third-parties): 3562 (23%)
• ‘Job Warehouse’ postings: 213 (1%)
• ‘Company Confidential’ postings: 2328 (15%)
• Appear to be ‘apprenticeships’: 423 (3%)
• Appear to involve ‘catalogue distribution’: 1168 (8%)
• Part-time: 3358 (22%)
• Postings declared self-employed: 1798 (12%)

This is somewhat like earlier attempts found elsewhere on the Blog. It’s early days, for this examination of Universal Jobmatch, and it may be expanded in future weeks. In examining the data we’ll try not to be judgemental and leave conclusion-drawing and ranting to others. In spite of this your feedback is appreciated, as ever.

This post is automated, but there’s a Human of some description behind it. E-mail him to express your disappointment in this.

• Jobs posted last week (Week Number 18): 19265
• Vacancies were still live this week: 18218 (95%)
• ‘Agency’ postings (from third-parties): 5289 (27%)
• ‘Job Warehouse’ postings: 1265 (7%)
• ‘Company Confidential’ postings: 3079 (16%)
• Appear to be ‘apprenticeships’: 476 (2%)
• Appear to involve ‘catalogue distribution’: 1419 (7%)
• Part-time: 3613 (19%)
• Postings declared self-employed: 2351 (12%)

This is somewhat like earlier attempts found elsewhere on the Blog. It’s early days, for this examination of Universal Jobmatch, and it may be expanded in future weeks. In examining the data we’ll try not to be judgemental and leave conclusion-drawing and ranting to others. In spite of this your feedback is appreciated, as ever.

This post is automated, but there’s a Human of some description behind it. E-mail him to express your disappointment in this.

After the expiry date

where date(noted) < date(‘now’) – 28
and date(noted) > date(‘now’) – 57

It was noted that there were 64,696 postings in this month. The re-scrape has seen that 36,392 of them are no longer to be found on Universal Jobmatch (as of 2013-05-04). About 56% of the advertised vacancies can therefore be though ‘filled’ after one month. But what about the rest? After all, these vacancies, 28,304 of them (43%) have not been filled, and it constitutes a fairly poor success rate. Let’s examine the postings …

• Agency postings: 5130
• Company confidential: 5679
• Catalogue postings: 2931

Top non-Company confidential posters with month-old live vacancies:

For those too lazy for Google, all the non-Company Confidential posters seem to be Kleeneze or similar ‘distributors’, except for All Answers who provide essay writing for the financially gifted but intellect-challenged student.

One can argue of these postings must therefore be ‘perpetual’ in that staff turn-over is high enough to always know that replacements are required. It also suggests that the same vacancies are constantly being re-posted so that they appear toward the top of searches. This is scope for future investigation.

It is all rather depressing, though, and when one reads a boast of how many vacancies there are on Universal Jobmatch (usually in total, or with a specific geography) you need to remember this, and take such pronouncements with a pinch or more of salt.

A computer definitely too old

Well I’m just about to finish here. I’ve spent most of the day fiddling with an ancient machine (called Orange, another from the McDonald’s skip). It’s wheezing mess with Ubuntu 10.04 imposed upon it, but impossible with 12.04. Not enough memory for all the pretty animated widget tripe.

I’ve therefore put it up under the server and install LXDE as it’s Window Manager. It’s usable again.

There was something called Lubuntu, a supposed Light-weight Ubuntu, but I didn’t have enough memory for the graphical install, and the server solution seemed the easiest way.

I may write this up and put it on the Blog. All appropriately pointless, and therefore apt.

Inspired by a post on a popular Tech News site, I have.

Although a FreeBSD fan, the bulk of these machines, elderly though they are, are under the Linux/GNU distribution known as Ubuntu, and the last Long Term Serviced (LTS) release 10.04 (also know as LucidLynx). As this was getting long in the tooth, it was decided that it would be prudent to upgrade. The standard upgrade path was attempted but largely because of the antiquity of the hardware, ended in abject failure.

After some experimentation an install the Ubuntu LTS server version from CD, followed by modification was found to be the easiest mechanism. This is a non-graphical install, which can be used on limited-resource machines, with a download-able ISO is on ubuntu.com. Although a ‘server’ version is installed, none of the ‘standard’ package bundles needs to be installed bar SSH-Server, required to allow a remote terminal. A working network connection (usually wired Ethernet) is therefore assumed.

SSH is required, for when when booted, the display may not work due to odd Display choices made for the boot-screens. It is therefore necessary to login using ssh(1) and effect the following changes (as root) to /etc/default/grub:

 GRUB_CMDLINE_LINUX="nomodeset vga=normal"
 GRUB_GFXMODE=1024×768

Then uncommented this line:

 GRUB_TERMINAL=console

Saved the file and then run update-grub(8). This all noted by Jon Witts, and possibly others, by the way.

X-Windows needs to be installed from the package xserver-xorg. A series of apt-get(8) commands can then be used to set the machine up to taste.

The author is a dinosaur and has set up twm(1) and xdm(8). A set of standard tools was also installed, including the beloved emacs, and the necessary gnucash and so forth.

Using twm seemed a lot better and more responsive than even Gnome. Twm needed a hack, LC_CTYPE set to en_US or fonts were screwed up. This finding by Andre Majorel writing in FixUnix.com, and we’re grateful to him. The edit needs to be made to an user’s .xsession file. LC_CTYPE seems to be unassigned in other X11 tools and this has no impact on them. It was also noted that Emacs’ resource string appears to now be ‘emacs’ rather than ‘Emacs’. Thus an edit made to .Xresources.

While twm proved to be fine for some curmudgeons, other more ordinary users wanted something akin to the existing Gnome desktop. Attempts to get an ‘old’ Ubuntu desk-top were abandoned after several attempts. To be honest the old Gnome based desktop left these machines somewhat slow and breathless too.

After careful consultation of the Internet, it was decided that LXDE would provide the best, light but usable, interface. And this was thus installed using apt-get, once again. One or two tweaks were needed, the default gaudy wall-paper was replaced, and the now-classic Clearlooks theme installed. Otherwise it was fairly straightforward. It is suggested that the Synaptic Package Manager be explored, if one is meandering down the graphical route.

As you may be aware, all this now looked like a ‘canned’ Lubuntu distribution. The Lubuntu distribution was tried; it is Ubunutu 12.04 LTS with an LXDE interface. Sadly the graphic install of this ‘lightweight’ version of Ubuntu stalls, as it runs out of memory. There is a non-graphics install of this distribution but installation was not attempted. Both version of Lubuntu also tends to install things that, while they have merit, are not required. Abiword the smaller-than-Open Office word-processor is cited as an example.

In summary, then, by installing Ubuntu 12 LTS server and then installing appropraite graphics packages a workable system can be produced on what is now extremely ancient hardware. It is realised that Sound and WiFi problems have yet to be addressed, and that there are still limitations from the hardware in speed of calculation on Spreadsheets and the display of video, as obvious examples.

Tinkering continues, on an on and off basis, and should further developments occur, they’ll be reported here. Your experiences should be shared too, so the usual exhortations to feedback are made.

• Jobs posted last week (Week Number 17): 17493
• Vacancies were still live this week: 16587 (95%)
• ‘Agency’ postings (from third-parties): 6601 (38%)
• ‘Job Warehouse’ postings: 446 (3%)
• ‘Company Confidential’ postings: 3276 (19%)
• Appear to be ‘apprenticeships’: 506 (3%)
• Appear to involve ‘catalogue distribution’: 1385 (8%)
• Part-time: 3650 (21%)
• Postings declared self-employed: 2228 (13%)

This is somewhat like earlier attempts found elsewhere on the Blog. It’s early days, for this examination of Universal Jobmatch, and it may be expanded in future weeks. In examining the data we’ll try not to be judgemental and leave conclusion-drawing and ranting to others. In spite of this your feedback is appreciated, as ever.

This post is automated, but there’s a Human of some description behind it. E-mail him to express your disappointment in this.

• Jobs posted last week (Week Number 16): 16633
• ‘Job Warehouse’ postings: 184 (1%)
• ‘Company Confidential’ postings: 2870 (17%)
• Appear to be ‘apprenticeships’: 327 (2%)
• Part-time: 3469 (21%)
• Postings declared self-employed: 2053 (12%)

This is somewhat like earlier attempts found elsewhere on the Blog. It’s early days, for this examination of Universal Jobmatch, and it may be expanded in future weeks. In examining the data we’ll try not to be judgemental and leave conclusion-drawing and ranting to others. In spite of this your feedback is appreciated, as ever.

This post is automated, but there’s a Human of some description behind it. E-mail him to express your disappointment in this.

Update: The self-declared ‘Job Warehouse’ classification is declining. This designation was being used by agencies posting other agency vacancies. The reality of these vacancies and their ultimate origin, a potential employer, was moot. However, this practise is still going on, just they’re not from ‘Job Warehouses’ any more. So it is intend to replace it with a general ‘Agency’ classification next week. The definition of ‘Agency’ being a entity posting on behalf of some anonymous third-party, and is somewhat arbitrary. Your feedback would be appreciated.

The measurement of still-current vacancies has started. This will appear as a crude measurement here too. 2013-04-22