Life as a Web developer can be hard when things start going wrong. The problem could be in any number of places. Is there a problem with the request you’re sending, is the problem with the response, is there a problem with a request in a third party library you’re using, is an external API failing?

Good tools are invaluable in figuring out where problems lie, and can also help to prevent problems from occurring in the first place, or just help you to be more efficient in general. Command line tools are particularly useful because they lend themselves well to automation and scripting, where they can be combined and reused in all sorts of different ways. Here we cover six particularly powerful and versatile tools which can help make your life a little bit easier.

(Image credit: kolnikcollection)

Curl

Curl is a network transfer tool that’s very similar to Wget, the main difference being that by default Wget saves to a file, and curl outputs to the command line. This makes it really simple to see the contents of a website. Here, for example, we can get our current IP from the ifconfig.me website:

$ curl ifconfig.me
93.96.141.93

Curl’s -i (show headers) and -I (show only headers) options make it a great tool for debugging HTTP responses and finding out exactly what a server is sending to you:

$ curl -I news.ycombinator.com
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Cache-Control: private
Connection: close

The -L option is handy, and makes curl automatically follow redirects. Curl has support for HTTP Basic authentication, cookies, manually setting headers and much, much more.

Ngrep

For serious network packet analysis there’s Wireshark, with its thousands of settings, filters and configuration options. There’s also a command-line version, TShark. For simple tasks I find Wireshark can be overkill, so unless I need something more powerful, ngrep is my tool of choice. It allows you to do with network packets what grep does with files.

For Web traffic you almost always want the -W byline option, which preserves linebreaks, and -q is a useful argument which suppresses some additional output about non-matching packets. Here’s an example that captures all packets that contain GET or POST:

ngrep -q -W byline "^(GET|POST) .*"

You can also pass in additional packet filter options, such as limiting the matched packets to a certain host, IP or port. Here we filter all traffic going to or coming from Google, using port 80 and containing the term “search.”

ngrep -q -W byline "search" host www.google.com and port 80

Netcat

Netcat, or nc, is a self-described networking Swiss Army knife. It’s a very simple but also very powerful and versatile application that allows you to create arbitrary network connections. Here we see it being used as a port scanner:

$ nc -z example.com 20-100
Connection to example.com 22 port [tcp/ssh] succeeded!
Connection to example.com 80 port [tcp/http] succeeded!

In addition to creating arbitrary connections, Netcat can also listen for incoming connections. Here we use this feature of nc, combined with tar, to very quickly and efficiently copy files between servers. On the server, run:

$ nc -l 9090 | tar -xzf -

And on the client:

$ tar -czf dir/ | nc server 9090

We can use Netcat to expose any application over the network. Here we expose a shell over port 8080:

$ mkfifo backpipe
$ nc -l 8080  0<backpipe | /bin/bash > backpipe

We can now access the server from any client:

$ nc example.com 8080
uname -a
Linux li228-162 2.6.39.1-linode34 ##1 SMP Tue Jun 21 10:29:24 EDT 2011 i686 GNU/Linux

While the last two examples are slightly contrived (in reality you’d be more likely to use tools such as rsync to copy files and SSH to remotely access a server), they do show the power and flexibility of Netcat, and hint at all of the different things you can achieve by combining Netcat with other applications.

Sshuttle

Sshuttle allows you to securely tunnel your traffic via any server you have SSH access to. It’s extremely easy to set up and use, not requiring you to install any software on the server or change any local proxy settings.

By tunneling your traffic over SSH, you secure yourself against tools like Firesheep and dsniff when you’re on unsecured public Wi-Fi or other untrusted networks. All network communication, including DNS requests, can be sent via your SSH server:

$ sshuttle -r <server> --dns 0/0

If you provide the --daemon argument, sshuttle will run in the background as a daemon. Combined with some other options, you can make aliases to simply and quickly start and stop tunneling your traffic:

alias tunnel='sshuttle --D --pidfile=/tmp/sshuttle.pid -r <server> --dns 0/0'
alias stoptunnel='[[ -f /tmp/sshuttle.pid ]] && kill `cat /tmp/sshuttle.pid`'

You can also use sshuttle to get around the IP-based geolocation filters that are now used by many services, such as BBC’s iPlayer, which requires you to be in the UK, and Turntable, which requires you to be in the US. To do this, you’ll need access to a server in the target country. Amazon has a free tier of EC2 Micro instances that are available in many countries, or you can find a cheap virtual private server (VPS) in almost any country in the world.

In this scenario, rather than tunneling all of our traffic, we might want to send only traffic for the service we are targeting. Unfortunately, sshuttle only accepts IP address arguments and not hostnames, so we need to make use of dig to first resolve the hostname:

$ sshuttle -r <server> `dig +short <hostname>`

Siege

Siege is a HTTP benchmarking tool. In addition to load-testing features, it has a handy -g option that is very similar to curl’s -iL, except it also shows you the request headers. Here’s an example with Google (I’ve removed some headers for brevity):

$ siege -g www.google.com
GET / HTTP/1.1
Host: www.google.com
User-Agent: JoeDog/1.00 [en] (X11; I; Siege 2.70)
Connection: close

HTTP/1.1 302 Found
Location: http://www.google.co.uk/
Content-Type: text/html; charset=UTF-8
Server: gws
Content-Length: 221
Connection: close

GET / HTTP/1.1
Host: www.google.co.uk
User-Agent: JoeDog/1.00 [en] (X11; I; Siege 2.70)
Connection: close

HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
X-XSS-Protection: 1; mode=block
Connection: close

What Siege is really great at is server load testing. Just like ab (an Apache HTTP server benchmarking tool), you can send a number of concurrent requests to a site, and see how it handles the traffic. With the following command, we’ll test Google with 20 concurrent connections for 30 seconds, and then get a nice report at the end:

$ siege -c20 www.google.co.uk -b -t30s
...
Lifting the server siege...      done.
Transactions:                    1400 hits
Availability:                 100.00 %
Elapsed time:                  29.22 secs
Data transferred:              13.32 MB
Response time:                  0.41 secs
Transaction rate:              47.91 trans/sec
Throughput:                     0.46 MB/sec
Concurrency:                   19.53
Successful transactions:        1400
Failed transactions:               0
Longest transaction:            4.08
Shortest transaction:           0.08

One of the most useful features of Siege is that it can take a file of URLs as an input, and then hit those URLs rather than just a single page. This is great for load testing, because you can replay real traffic against your site and see how it performs, rather than just hitting the same URL again and again. Here’s how you would use Siege to replay your Apache logs against another server to load test it:

$ cut -d ' ' -f7 /var/log/apache2/access.log > urls.txt
$ siege -c<concurrency rate> -b -f urls.txt

Mitmproxy

Mitmproxy is an SSL-capable, man-in-the-middle HTTP proxy that allows you to inspect both HTTP and HTTPS traffic, and rewrite requests on the fly. The application has been behind quite a few iOS application privacy scandals, including Path’s address book upload scandal. Its ability to rewrite requests on the fly has also been used to target iOS, including setting a fake high score in GameCenter.

Far from only being useful to see what mobile applications are sending over the wire or for faking high scores, mitmproxy can help out with a whole range of Web development tasks. For example, instead of constantly hitting F5 or clearing your cache to make sure you’re seeing the latest content, you can run

$ mitmproxy --anticache

which will automatically strip all cache-control headers and make sure you always get fresh content. Unfortunately it doesn’t automatically set up forwarding for you like sshuttle does, so after starting mitmproxy you still need to change your system-wide or browser-specific proxy settings.

Another extremely handy feature of mitmproxy is the ability to record and replay HTTP interactions. The official documentation gives an example of a wireless network login. The same technique can be used as a basic Web testing framework. For example, to confirm that your user signup flow works, you can start recording the session:

$ mitmdump -w user-signup

Then go through the user signup process, which at this point should work as expected. Stop recording the session with Ctrl + C. At any point we can then replay what was recorded and check for the 200 status code:

$ mitmdump -c user-signup | tail -n1 | grep 200 && echo "OK" || echo "FAIL"

If the signup flow gets broken at any point, we’ll see a FAIL message, rather than an OK. You could create a whole suite of these tests and run them regularly to make sure you get notified if you ever accidentally break anything on your site.

(cp)

---

© Ben Dowling for Smashing Magazine, 2012.

UICloud is a fresh project. Launched in July 2012 it already provides over 25,000 UI Elements in over 1,000 UI sets. All the resources can be used free of charge, individual licenses apply. Jack Cai, a Southampton (UK) based Chinese freelance icon and GUI designer, did a great job. The platform is intuitive, even fun to use. The design is modern and elegant, the content is neatly arranged. Enough reason to look deeper into it.

Masking passwords is an old practice that’s commonly implemented in sign-up and log-in forms. It’s used to prevent over-the-shoulder snoopers from catching the user’s password. While masking passwords is a good security practice, there’s a chance it could jeopardize the user experience of your sign-up form. When users sign up on a website, they expect a no-hassle, worry-free form to fill out. But masking the password could prevent that.

Good For Logging In, Bad For Signing Up

Log-in forms are used more often than sign-up forms. Users only need to sign up once to create an account, whereas they will need to log in multiple times to access their account. Because log-in forms are used so frequently, there’s a strong chance that users will end up typing their password in front of other people. Users sometimes want to show their friends or colleagues something on the website, and they would need to log in to do so. Therefore, masking passwords in log-in forms is good because it keeps passwords hidden every time the user logs in.

However, masking passwords in sign-up forms is different. Password masking generally causes users to make more typing errors because they can’t see what they’re typing and can’t tell whether they’ve made a mistake. The consequences of making a typing error when logging in are not as serious as making one when signing up. If the user fails to type in the right password when logging in, they simply try again. If they type in the wrong password when signing up, they’ll get locked out of their account when they try to log in and will have to reset their password. The user isn’t to blame when this happens. It’s the designer’s fault for not making it easy for the user to see what they’re typing in the password field.

What If We Omit Confirmation Fields In Sign-Up Forms?

A big hurdle that password masking creates for users is the password-confirmation field commonly found in sign-up forms. This field requires users to retype their password and checks that both match so that the wrong password doesn’t go through. The reason why password-confirmation fields exist is that users sometimes make typos when typing their password with masking on, and this extra field can catch those typos.

Password-confirmation fields might be well intentioned, but they have a downside. Users are prone to making even more typos because they have to type their password twice in separate fields with masking on. What’s worse is the extra work they have to do to correct their typos; because they can’t see where their typos are, users have to clear the fields entirely and retype their password. The password-confirmation field not only causes more typos, but forces users to do more work to fix them, thus slowing users down and making sign-ups more of a pain.

Unmasking Passwords Temporarily Decreases Typos

Masking passwords in sign-up forms might give users more trouble than it’s worth. It masks not only the password, but any typos the user makes, making them hard to spot and fix. The security it provides is often less than helpful because many people usually sign up for websites in private, with no one looking over their shoulder. Signing up is usually a one-time deal; once they’ve done it, they don’t need to do it again. Displaying their password in plain text that one time when they are alone is probably not as a significant security risk as we tend to think. The chance of a snooper catching the password is slim to none, even if the user is signing up in public, because most snoopers aren’t random strangers, but rather someone the user knows.

The solution to all of these issues is to temporarily unmask the password so that the user can fill in the field quickly and accurately — i.e. unmasking the password for a moment so that the user can see what they’ve typed. Temporary unmasking decreases typos and makes it easy for users to catch and fix any typos they make. And the user doesn’t have to worry about snoopers stealing their password because the unmasking is quick, e.g. if we unmask a couple of last characters typed in. Snoopers would have to memorize a string of (hopefully) random alphanumeric characters in a matter of seconds, which is very hard to do. If we unmask only last characters, they would need to look over the shoulders for a longer period of time to be able to “catch” the whole pass phrase.

I strongly believe that a lot of the snooping paranoia is in our minds — the bigger issue is users getting locked out of their account because of typos caused by masked passwords. Below are a couple of simple techniques to prevent that from happening.

Unmasking on Field Focus

You can make the password field easy to fill in and secure at the same time by unmasking the password when the keyboard focus is on the field and then automatically masking it when the focus is off the field. This allows the user to see the characters they’re typing only when the password field is selected, thus decreasing the risk of typos and preventing others from sneaking a peek when the user has moved on to other fields.

Another small security measure you could add is to display the user’s password in small, light-gray italicized text. Thus, being able to make out each character would require moving close to the screen. In the unlikely event that a snooper is looking on, the modified text would make the password indiscernible to everyone except the person sitting right in front of the screen.

Another option would be to display only the last characters of the password while hiding other characters with asterisks, thus confirming the user’s input as the password is typed in.

Unmasking With a Checkbox

Another approach is to provide a checkbox for unmasking. Thus, when the user types their password, it is masked, but when they check the box, it gets unmasked, allowing them to see whether they’ve made a typo. A little more effort is required with this approach with the checking and unchecking, but it’s far better than a password-confirmation field because it enables users to see and fix their typos with ease.

Balancing Security And User Experience

Following design conventions is generally advisable, but when a convention slows users down, complicates a task or increases the chance of error, it needs serious reconsideration. Security should be balanced with the user experience. Favor security too much over the experience and you’ll make the website a pain to use. Favor the experience too much over security and you’ll make visitors nervous about using the website. When you find that balance, users won’t have any trouble using your website, even if it doesn’t adhere to every design convention.

(al)

---

© Anthony T for Smashing Magazine, 2012.

Roles have been an integral part of WordPress for quite some time now — many functions associated with managing them have been in place since version 2.0.0. Despite this longevity, they are rarely utilized which is a shame since they allow for the easy setup of custom user types (and also have the ability to micro-manage them). In this article, you’ll learn everything you need to utilize user roles in WordPress and make the most of this incredible built-in functionality.

Large view.

Terminology

The main terms to know are “role” and “capability”. Capabilities are the core of the system, they represent the things you can do. An example of a capability is switch_themes. A user who has this capability is able to change the look of the website using the Appearances section in the admin. Those who do not have this capability will not be able to do so.

Roles are simply a named set of capabilities. Imagine you have capabilities like edit_post, add_user and so on. Instead of listing all 50 of them for each and every user who you’d like to be an admin, simply assign each user the “admin” role and then assign all the capabilities to that role.

The Default Setup

By default, WordPress has six roles and about 57 capabilities. Each role has a different combination of capabilities which endow the user with the appropriate rights and privileges.

For example, a contributor can edit and delete his own posts, but cannot control when these posts are published (and cannot delete those posts once they are published). This is because they have the edit_posts and delete_posts capabilities, but they do not have the publish_post and delete_published_post capabilities.

The default setup offered by WordPress is very well thought out and should not be modified, only added to. If it is changed, you could face difficult problems later on — like a contributor being able to remove posts from your website, or an admin not being able to change the theme.

When New Roles And Capabilities Are Needed

New roles usually come hand-in-hand with new capabilities. Usually, we first create a set of new capabilities, which are held by the admin (and a new role, as well). Let’s look at an example.

If you have a large website, chances are you have a marketing team. This team doesn’t need to be able to edit and publish posts, but they do need access to advertising stats, trending search topics, etc. Perhaps it would also be beneficial to allow them to manage categories and comments for SEO purposes and customer satisfaction, respectively.

Our first order of business is to look at the advertising stats and trending searches. This is not something WordPress offers by default, so let’s presume it has been built as a plugin. Let’s create a capability called view_stats. This will be given to the marketing people and of course to the admin. Other users — like editors and authors — do not need to see the stats (which, in many case, is sensitive data) so they will not receive this capability.

Our second task is to decide what other capabilities our marketing people need. Based on our assumptions above, they will need the read, manage_links, manage_categories and moderate_comments capabilities.

Lastly, we would use some coding chops to create a new role named “Marketing�, and assign the above-mentioned capabilities to it. Once that has been done, we are able to appoint the “Marketing� role to any user within the “Add User� and “Edit Profile� page.

Basic WordPress Functions

To manage roles and capabilities effectively, all you need is five very straightforward functions:

• add_role() — Enables you to add a custom role
• remove_role() — Enables you to remove a custom role
• add_cap() — Enables you to add a custom capability to a role
• remove_cap() — Enables you to remove a custom capability from a role
• get_role() — Gets information about a role as well as associated capabilities

If you would like to view the documentation for these functions, you can take a look at the Roles and Capabilities section in the Codex.

Implementing New Roles And Capabilities

The full implementation of roles and capabilities requires more than just the five functions I mentioned. We’ll also need to use functions for checking capabilities, we’ll need features which are only available to certain roles, and so on.

Adding the Roles and Capabilities

$marketing = add_role( 'marketing', 'Marketing', array(
    'read' => true,
		'manage_links', 
    'manage_categories', 
    'moderate_comments',
    'view-stats'
));

$role = get_role( 'administrator' );
$role->add_cap( 'view-stats' );

The function above will add our new “Marketing” role. It adds four capabilities which are already included in WordPress and a fifth one which is a custom capability. As you can see, no special coding is required to add custom caps — just include them in the array when adding the role. Once we add the new role, we make sure to add our new capability to the admin role as well.

Note that the add_role() fuction returns a WP_Role object on success or NULL if the role already exists.

Since the above code will be used in a plugin, the best way to use it is to utilize hooks, and to add the capabilities to the administrator only when we first create the role.

add_action( 'admin_init', 'add_custom_role' );

function add_custom_role() {
    $marketing = add_role( 'marketing', 'Marketing', array(
        'read' => true,
	    	'manage_links', 
        'manage_categories', 
        'moderate_comments',
        'view-stats'
    ));

		if( null !== $marketing ) {
        $role = get_role( 'administrator' );
        $role->add_cap( 'view-stats' );
    }
}

Checking for Capabilities and Roles

Once we have our system in place we can check for capabilities and roles. This allows us to make sure that only users with the right capabilities can do certain things.

add_action('admin_menu', 'register_stats_Page');

function register_stats_Page() {
    add_menu_page( 'Marketing Info', 'Marketing Info', 'view_stats', 'marketing-info', 'show_marketing_info' );
}

function show_marketing_info(){
   if( ! current_user_can( 'view-stats' ) ) {
      die( 'You are not allowed to view this page' );
   }
   echo "
This is the marketing stats page
";	
}

The code above will add a top level menu to the admin area for our marketing stats page. The add_menu_page() function allows you to specify a capability (the third argument) which determines whether or not the menu is shown.

Note that even though the add_menu_page() function itself takes the capability as an argument, we still need to check for it in the function which displays the content of the page.

This leads us into our next two functions nicely: current_user_can() and user_can().

If you want to check whether or not the currently logged-in user has a specific capability, simply use the following format:

if( current_user_can( 'any_capability' ) ) {
    // The user has the capability
}
else {
    // The user does not have the capability
}

If you would like to check whether or not an arbitrary user has a specific capability, use the user_can() function:

if( user_can( 45, 'any_capability' ) ) {
    // The user with the ID of 45 has the capability
}
else {
    // The user with the ID of 45 does not have the capability
}

In addition to capabilities, both fucntions can check for roles as well:

if( current_user_can( 'marketing' ) ) {
    // The current user is a marketing person
}
else {
    // The current user is not a marketing person
}

Creating User Types

While not an official term, I call different roles “user types” when they require many different front-end changes. Say you’re building a theme marketplace, for example. Apart from the administrator who oversees all (and the authors who would write in the company blog) you also have buyers and sellers. These are both registered users of your website but would require somewhat differing front-ends.

The profile page of a buyer would most likely show the number of things he’s bought, how frequently he buys stuff and when logged in, he would see a list of downloads.

A seller’s profile would be a portfolio of his work, his most popular items, and so on. When logged in he should probably see sales statistics.

Figuring Out Roles and Permissions

To make things simple, users either buy or sell, and cannot do both. Let’s create the two roles with a set of capabilities (off the top of my head…):

add_action( 'admin_init', 'add_custom_roles' );

function add_custom_roles() {
    $seller = add_role( 'seller', 'Seller', array(
        'read' => true,
	    	'delete_posts' => true,
	    	'edit_posts' => true,
	    	'delete_published_posts' => true,
	    	'publish_posts' => true,
	    	'upload_files' => true,
	    	'edit_published_posts' => true,
        'view_sales' => true,
        'moderate_comments' => true
    ));

    $role = get_role( 'administrator' );

		if( null !== $seller ) {
        $role->add_cap( 'view_sales' );
        $role->add_cap( 'view_others_sales' );
   }

    $buyer = add_role( 'buyer', 'Buyer', array(
        'read' => true,
        'add_payment' => true,
        'download_resources' => true
    ));

		if( null !== $buyer ) {
        $role->add_cap( 'add_payment' );
        $role->add_cap( 'add_others_payment' );
        $role->add_cap( 'download_resources' );
        $role->add_cap( 'download_others_resources' );
   }

}

The idea here is that sellers are like authors, who will store sellable items as posts. Therefore, they receive the capabilities of an author. In addition, they can also view sales stats for their own items.

In addition to viewing items, buyers can also add payments and download files related to items that they buy.

Admins receive all these permissions, but in addition, they may view the statistics of any other user, add payments to any account, and may download resources for any item.

Frontend Implementation

From here on out it’s a matter of using our role and capability checking functions to determine who sees what. Let’s take a look at a couple of examples:

Listing Users

If we list all the users on the website, we could show the items bought for the buyers and the items sold for the sellers. Here’s how:

<!-- 
	User Data comes from a WordPress user object stored in the $user variable 
--> 

<div class='user'>
	<div class='row'>
		<div class='threecol'>
			<?php echo get_avatar( $user->ID, 120 ) ?>
		</div>
		<div class='sixcol'>
			<h1><?php echo $user->display_name ?></h1>
			<?php echo $user->description ?>
		</div>
		<div class='threecol last'>
			<?php if( user_can( $user->ID, 'buyer' ) ) : ?>	
				<div class='counter'>
					<span class='number'><?php get_purchases_count( $user->ID ) ?></span>
					<span class='text'>purchases</span>
				</div>
			<?php else ?>
				<div class='counter'>
					<span class='number'><?php get_sales_count( $user->ID ) ?></span>
					<span class='text'>sales</span>
				</div>			
			<?php endif ?>
		</div>
	</div>
</div>

Most of this is plain old HTML, but the important thing to take away is how the counter is created. Note that the get_purchases_count() and get_sales_count() functions are fictitious (they are not part of WordPress, of course).

Of course, in a real world example we would need to make sure that only buyers and sellers are listed, and we might need to create a separate template for them completely (if the differences are numerous enough).

foreach( $users as $user_id ) {
    $user = get_userdata( $user_id );
    if( user_can( $user->ID, 'buyer' ) ) {
        get_template_part( 'userlist', 'buyer' );
    }
    else {
        get_template_part( 'userlist', 'seller' );
    }
}

When looping through our users above, either userlist-buyer.php or userlist-seller.php will be used, depending on the user’s role.

Conclusion

User roles and capabilities are extremely powerful tools in WordPress. Whenever you build a plugin which requires the implementation of special user-based functions, you should consider using them.

Apart from making your life easier (and more manageable) it will make the processes on your websites easier to follow and easier to backtrack any problems.

(jvb) (il) (jc) (js)

---

© Daniel Pataki for Smashing Magazine, 2012.

Image- or, more general, content sliders are as popular as can be. There are plenty of them on the market. Design and functionality resemble, the choice of a slider is more or less a matter of taste. Tiny Circleslider is different though. The tool lets you place content elements in a circle. This makes for a futuristic look and feel.