Wednesday, December 28, 2011

Cheap, easy way to make your dev server available on the public internet.


While solutions like localtunnel and showoff.io allow you to do this, they have some limitations both in terms of cost and functionality. The biggest problem is that it gets expensive for lots of developers or you can't use your own hostnames.

We've developed a alternative using DNS, a reverse proxy and an ssh tunnel that makes it trivial to allow public access to a any number of dev servers on demand.

Here's how it works:
  • Set up a reverse proxy (you can use Apache or nginx).
# assume public IP of 1.2.3.4 # need name-based virtual hosting # so we can support many dev boxes with a single IP NameVirtualHost 1.2.3.4:80 # Need a VirtualHost container for each developer <VirtualHost 1.2.3.4:80> ServerName jason.dev.domain.com ProxyRequests Off ProxyPreserveHost On ProxyPass / http://localhost:2000/ ProxyPassReverse / http://localhost:2000/ </VirtualHost> <VirtualHost 1.2.3.4:80> ServerName tim.dev.domain.com ProxyRequests Off ProxyPreserveHost On ProxyPass / http://localhost:2001/ ProxyPassReverse / http://localhost:2001/ </VirtualHost>
  • Configure a wildcard CNAME record for *.dev.domain.com that points to your proxy server. Using the wildcard avoids having to munge DNS for every new developer.
  • Set up a proxy user account on the box and add all developers' ssh keys to the account. All this user needs to do is to log in and forward non-privileged ports, so it can be locked down substantially.
  • Edit your /etc/hosts to so that the canonical name for your server points to your local dev IP.
# /etc/hosts entry 33.33.33.11 jason.dev.domain.com
  • To make your dev server publicly available, create an SSH tunnel. Remember that each developer will have a particular remote port number assigned to them and them only.
ssh proxy@proxy.dev.domain.com -R 2000:jason.dev.domain.com:80
  • This setup allows the exact same host name to be used everywhere but have it hit the local dev box locally and have the same name resolve to the public proxy for development that require it.
While this does require you to have a publicly-reachable server somewhere to configure the proxy, this probably isn't a huge problem for most companies. In return you get a near foolproof setup for debugging webhooks, mobile apps, etc, at no cost, and without jumping through any hoops or relying on any third-party systems.

Monday, November 28, 2011

Time Machine backups to a drive on another Mac over the network

I've got a huge external drive on my desktop Mac which I use for a Time Machine backup. However, my MacBook has gone un-backed up for a long time. I had heard over the years that you could hack your way to a network Time Machine backup, but never bothered trying (I try to avoid hacks because they usually blow up in my face and takes a ton of time to fix). However, now that I'm using Aperture heavily on my laptop, I don't want to go without a backup of all of my pics.

Turns out, it's very, very easy to set up an external drive on one Mac as a Time Machine volume over the network.
  1. Share the external hard drive. To do this, do a Get Info on the external drive and click the Shared Folder checkbox.
  2. Open up System Preferences > Sharing.
    1. Make sure File Sharing is enabled.
    2. You should see the external hard drive listed in the Shared Folders column. Click it.
    3. Click the + icon in the Users column and add the user you connect with from the networked computer.
    4. Set up Read & Write access for that user.
    5. Ideally you should remove/lower privileges for other users here. In my case the shared external drive had all kinds of unreasonable default permissions for Unknown User, Guest, and Everyone.
  3. On the networked machine, go to the terminal and enter:
    defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
  4. Use the Finder to connect to the machine hosting the external drive with the user you configured in step #3 above.
  5. Open System Preferences > Time Machine.
  6. Click Select Disk.
  7. You should see the external drive listed, pick it.
  8. Profit!
So next you're probably wondering, dang this initial backup is going to take forever. Well, there's a trick for that, too. Apple File Sharing works over a local non-routed network (ie Bonjour). So just hook up your two computers with an ethernet cable, and disable WiFi on the machine that doesn't have the external drive attached. You should still be able to see the shared external drive in the Finder, only now it's running on Gigabit Ethernet (or the fastest your two machines can manage with each other). This is a great way to do the initial backup, and subsequent backups will be much smaller and not as big of a deal.

Enjoy!

Saturday, November 19, 2011

Mouse support for Terminal.app: scrolling, vi/vim, and more!

Goofing around on the internet today just got real. I was reading about ncurses and noticed that the API supports mouse events. So wait, if terminals can support mouse events, then why doesn't vim in Terminal.app work like gVim and support mouse scrolling, clicking, and selection? It can, and it's AMAZING!

Someone wrote a SIMBL plugin called MouseTerm that passes through all mouse events to the terminal. After that, all you have to do is enable mouse support in vim, and Boom goes the Dynamite!

Steps:
1. Install SIMBL
2. Install MouseTerm
3. Edit your .vimrc:

" mouse support
if has("mouse")
    set mouse=a
    set ttymouse=xterm2
endif
4. That's it! Now you can use the mouse for clicking, selection, and scrolling!

One thing I did notice is that you can no longer copy text to the Mac clipboard from Vim in this mode. It's easy to toggle mouse event passing with Cmd-Shift-M (or menu item Shell > Send Mouse Events).

I used the ttymouse=xterm2 since it is the one recommended for use with gnu screen. ttymouse=xterm didn't behave properly for me when vim was used inside of screen.


Thanks to Ayaz for this article on "Using mouse inside Vim on Terminal.app" which got me pointed in the right direction.

Monday, November 07, 2011

Security Caveats with S3: it's easy to grant dangerous permissions with Bucket and User policies.


We've been exploring ways to use S3, and one of the ideas was to dynamically create buckets on-the-fly for individual customers to store and manage their objects.

Since we take security very seriously, we took a deep-dive into S3's permissions model to be sure that we could eliminate the risk of data loss due to application logic error and/or security breach of the web server.

As with any security setup, partitioning of access rights is key to risk mitigation. Thus we created an unprivileged webapp user that could not permanently delete S3 objects. Of course, we were trying to create buckets on-the-fly, so this same user would need the ability to create buckets and set bucket policies.

As it turns out, it's not possible to do this securely. This was a big shock to me, since what it means in practice is that UserA can grant rights to UserB even if UserA doesn't have those rights himself.

Steps:
1. Bucket Owner grants *only* PutBucketPolicy to UserA
2. UserA grants DeleteObject / DeleteObjectVersion to UserB.
3. UserB can now permanently delete every object in the bucket.

Even more shocking, this is also true if you've gone the extra step of enabling MFA-delete on the bucket. This means that you need an MFA device to successfully permanently delete objects. However, you don't have to have the MFA device originally used to enable MFA-delete. In the above scenario, even with MFA-delete enabled, UserB can buy his own MFA device, attach it to his account, and permanently delete any object in the bucket.

Furthermore, there is no place to attach deny rules that could not be similarly overturned, since the only place to attach such a rule is via a Bucket Policy, and of course that deny can be easily overwritten by the webapp user with the PutBucketPolicy permissions.

The bottom line is that giving PutBucketPolicy to a user is equivalent to giving them root access to the bucket. They can do anything. Thus, the AWS/S3 permission model is not currently suitable if you'd like your web application to be able to create and configure buckets on-the-fly.

I do not have enough of a security background to know if this is considered bad security architecture or just a normal caveat of complex permissions models, but I was personally very concerned that a user could grant permission he didn't have himself.

All of this has been confirmed through multiple experiments as well as with AWS Support. It is true. So be careful!

Saturday, April 09, 2011

Pearfarm: making it trivially easy to create and share php packages with PEAR

Jason Ardell and I had the honor of speaking at this month's Atlanta PHP user group. We gave a talk about the Pearfarm project, which I started in 2009 with a handful of other amazing PHP developers that were interested in bringing some community collaboration tools found in other language communities to the PHP universe.

The result was Pearfarm, which has 2 major features:
  1. Make it really easy to create PEAR packages
  2. Make it really easy to share php packages
That's it. Amazingly, this is not easy to do in the PHP community, even today, almost 2 years after we built Pearfarm. Yes, pyrus makes it easier, but there are caveats, like having to arrange your code in a certain directory structure. And even though they've made a PEAR channel server that's easy to run, well you still have to learn about PEAR channels and find a place to host a server with your PEAR channel. Seems like a lot of work to share a small package.

Pearfarm has been out for 18 months now, and while it is awesome at what it does and some people have started using it, it hasn't taken off. Mostly, I think, because we haven't pushed it very hard. But I think that we (the PHP community) should be pushing it harder. After all, it's for our own good.





One thing I learned from talking about pearfarm over the last couple of years that really shocked me is is how few people are using PEAR at all. If you ask php devs how they find packages the answer is google. It's not even the PEAR repository! To the average php developer on the street, PEAR is a dead project. I think largely this is due to the perception that the PEAR repository is so devoid of activity.

Many people have argued to me that having the PEAR repo is a good thing for the community since it provides packages that have been vetted at some level. I can accept that there is value in having that trust, but it comes at the expense of a rich, active developer ecosystem. It's too hard to participate in PEAR as a developer. I know the PEAR people will argue with me that I'm wrong and it's so easy, but I don't agree. It's hard to participate in PEAR as a contributor.

However, PEAR (the installer) is actually great! It makes it really easy to create local sandboxes of code libraries for each application. It handles dependencies. It has the tools built-in to create PEAR packages (though it's not trivial unless you use pearfarm or pyrus).

So I think we all owe it to the community to start using other people's code, and start sharing code. Pearfarm makes it seriously easy to share php packages. In no time at all we could completely re-invigorate the PHP community. Some of the bigger PHP projects are already starting to spin off a lot of quality php code. This is good, this is very good. EVERYONE INTO THE POOL!

Sunday, February 06, 2011

iCal Birthday Alarms


Apple often falls just short of making things really simple. A great example is the support for "Birthdays" in Mac OS X.

While the Address Book allows you to set a birthday for any contact, and you can then easily show all birthdays in iCal, they fail to provide a way to set an alarm for each birthday, so you can actually remember to call or send a card in advance. Nor are the events that are "imported" into iCal editable. Apple, you were *so close* to being helpful.

I've been searching for a simple solution to this forever, and thanks to this Applescript snippet I've created a solution I actually like.

As it turns out, though you cannot edit the alarms in the GUI, you *can* edit them via Applescript! The Applescript below will add a couple of alarms to every birthday in the "Birthdays" calendar (the default one Apple creates when adding the birthday calendar).

The bit I added to the original was the ability to add an email alarm as well. It's also good to note that this script is idempotent, meaning you can run it as often as you like (in fact you need to if you add a new birthday to the Address Book) and it will have the same result.

tell application "iCal"

tell calendar "Birthdays"

set all_events to every event

repeat with this_event in all_events

tell this_event

delete every display alarm

delete every sound alarm

delete every mail alarm

make new mail alarm at end with properties {trigger interval:-(7 * days) / (1 * minutes), email:"me@email.com"}

make new sound alarm at end with properties {trigger interval:(10 * hours) / (1 * minutes)}

end tell

end repeat

end tell

end tell

If you really want to automate this, you can save the Applescript as an application and add it to a cronjob so that you'll never forget a birthday again! Here's a snip of a cronjob that will update your birthday alarms weekly. My computer stays on all the time so the 5AM run time is no problem; if yours isn't always on I'd recommend changing the 5 to a more appropriate hour.
0 5 * * mon open ~/bin/Add\ Alarms\ to\ iCal\ Birthdays.app

Tuesday, November 02, 2010

Attention Lawyers: Here's a free-market solution to the problem that 2010 Georgia Amendment 1 tries to fix

When somebody wants to amend the state constitution, you figure there's been some horrible injustice that needs to be fixed right at the top. Today, you'll find the amendment on the 2010 Georgia ballot:

Shall the Constitution of Georgia be amended so as to make Georgia more economically competitive by authorizing legislation to uphold reasonable competitive agreements?
(__) Yes
(__) No

Sounds great, right? Who wouldn't want Georgia to be more competitive.

Unfortunately, the wording of the ballot question is a huge deception, and you should read more about why you should #VoteNoOn1 all over the internet.

From what I understand, the original impetus behind this proposal was to correct an actual problem that occurs in Georgia related to employment contracts.

From an AJC article about the amendment:

A Georgia businessman thought he had an ironclad non-compete agreement with a new employee he hired to handle sales at his company. The employee had access to confidential information about customers, and the agreement prohibited her from soliciting those customers.

But while the court found the confidentiality agreement was valid, it voided the non-compete agreement because a clause in another section didn't give a time frame for the non-compete agreement as required -- even though the time frame of two years was referenced in another clause, said Shawn Anton Kachmar, the lawyer who represented the businessman.

Clearly this is an unfortunate result for the businessman in question -- he thought ahead enough to create an employment contract, but he didn't know it might be overturned in court. Maybe he used a lawyer to write the agreement -- if so, this sounds a lot like legal negligence or malpractice. But trying to prevent the situation (an employer that had a poorly-written employment contract) above by changing the foundation of how employment contract law works in Georgia is not the way to do it. Rather, I propose a market-based solution.

When a company asks an employee to sign an employment contract, the core reason for the agreement is that the company doesn't want the employee to leave and either steal customers that he formed a relationship with or help a competitor advance their product. Both of these are pure monetary problems; if the employee breaks the employment contract, there are monetary damages that the company would suffer. The recourse for the company in the case of a broken employment agreement is probably monetary (although I suppose in some cases the courts could make the employee quit the new job, but I doubt the courts work fast enough for that to matter much). The trick here is that no one knows if the employment agreement will be held up in court until it goes to court. But the law firm that wrote the contract should have a very good idea of whether it will hold up.

What I propose is that law firms start offering employment contracts along with insurance. Instead of paying say $500 to draft an employment contract, the company would pay maybe $1000, and this would include some insurance. If the employee leaves and violates the employment contract, the law firm would sue the employee, and if the employment agreement is thrown out, the law firm would have to pay out a claim to the company. It's basically a little malpractice insurance wrapped up with an employment agreement. But it's a market solution that should have the effect of reducing the number of unenforceable employment contracts in the state while increasing profits for law firms. That sounds like a free market win-win to me.