[clue-talk] Assessing technical skills?

[Jeff Cann]

Greetings.

At work, I lead a team of web engineers.  Our definition of people like 
this is a system administrator who works mostly on software [as opposed 
to being a SA who is a hardware expert].  In particular, you need the 
ability to work in the web application realm.  Web servers, app servers, 
HTTP, SSL, FTP, J2EE, CGI are common protocols / standards that you need 
to know to be effective.

The buck usually stops with our team because are at the top [or bottom, 
depending on your point of view] of an application  [database, app 
server, web server, network, and hardware all combine to become the 
app].  It's not a good job for some people as too many things can break 
and you have to be able to identify [fast] whom to call if you can't fix 
it.  We like people who have development backgrounds because they have 
[usually] great troubleshooting / diagnostic / analysis skills.  We also 
like traditional Sys Admins because they know how to avoid risks 
[usually].  We also like people who prevent problems, rather than 
waiting for them to happen.

I've hired 2 contractors in the past bit and both are not doing very 
well.  Both had good resumes, but we relied on verbal interviews where 
we drilled them on past problems, solutions, etc.  I regret not asking 
for some type of written test / quiz because [based on performance] I 
think I assumed too much in the interviews.  It's clear that when I put 
5 years of UNIX as a requirement, people think 'I had UNIX in college' 
covers it.  In the end, they are useless at the command line, and this 
is where 95% of our work happens.

So, as I look for replacements, I'm wondering how do other people assess 
technical skills in an interview?  Obviously, seeing the person work on 
a Linux command line is the first clue.  But, I don't want to demean 
anyone by making them show me that they actually can use vi.  OTH - It 
seems obvious now that I have to have more evidence of their skills than 
what I get from traditional interviews / resume checks.

I appreciate any suggestions.

Jeff