Tuesday, March 19, 2013


OAuth - A great way to cripple your API



Intro

 A few years ago, the big social networking sites were looking for a secure way to allow their users to safely use any and all untrusted software to access their own personal accounts. So that a user could have their account on one social networking site safely interact with their account on another social networking site. They also wanted to allow for users to be able to allow their accounts to interact safely with various untrusted web applications and web sites.

In order to safely allow untrusted software to access a user's account, a few points needed to be kept in mind.
  • Untrusted software should not have access to a user's credentials, in case the software is compromised, passwords will not be stolen, as user's passwords are not stored by the software.
  • Untrusted software should not have full access to a user's account, but only limited access as defined by the user. In the same vein, giving the software user's personal credentials will allow unlimited access to a user's account, which could be used maliciously.
  • A user should be able to revoke the permission they granted to allow particular untrusted software to work, in case they turn malicious, despite the limited access they have.
A solution that was developed for this use-case was created - OAuth. OAuth's primary use-case is the one described above.

Implementation

OAuth is generally implemented in a fashion where the untrusted software is accessed by a user via a standard web browser. In order for a user to authorize that software, the software will redirect the user's browser to a page on the social networking site, with a couple of parameters sent to it. Now that the user is on his social networking site, the untrusted software no longer has control over what the user is doing, as the user left the untrusted software, allowing the user to safely log in to his social networking account. Then, the social networking site will see the parameters it was passed, and ask the user if he or she wants to authorize the software in question, and what kind of access to the user's account it should be given.

If and when the user has authorized the untrusted software, the social networking site can then report back to the untrusted software that it was granted a certain amount of access, and give it some credentials to use. These credentials are unique to the user and the software in question. The social networking site allows for a user to later on see a list of software authorized, and revoke the unique credentials given to any one of them, or modify the amount of access a particular set of credentials has.

Problems

There is no standard

Now above, I said OAuth is generally implemented in this fashion. I say generally, because unlike standard HTTP authentication schemes (Basic, Digest, and others), OAuth is a big grab bag of ideas which can be mixed and matched in an infinite amount of ways, and also allows for developers to make their own unique tweaks to their personal implementations.

With the standard HTTP authentication schemes, once a developer knows the protocol, and implemented it with one web site, that exact same knowledge can be reused to logging into any other web site that supports the standard. Likewise, software libraries can be made to handle HTTP authentication, and all a third party developer needs to do is specify to the library which credentials should be used, and then everything works as expected.

With OAuth, once a developer learns how to authenticate with it to one web site, it is likely that the same developer will need to relearn how to connect to every other web site using OAuth. This further means that every web site which supports OAuth needs to document exactly how it is implementing it, and what tweaks are in use. It also means that no library can be written which can simply support every OAuth implementation out there. This places a great burden on developers on both sides. It can also greatly increase frustration for less able users, when their favorite library works great with one site they support, but are unable to extend their software to another site, because their library lacks some aspect of OAuth for this new site, or a unique OAuth tweak on this new site renders the library incompatible.

Here are some choice quotes from the official RFC:
  • However, as a rich and highly extensible framework with many optional components, on its own, this specification is likely to produce a wide range of non-interoperable implementations.
  • This framework was designed with the clear expectation that future work will define prescriptive profiles and extensions necessary to achieve full web-scale interoperability.

Another issue is that while various standard HTTP authentication schemes have well understood security margins, OAuth is entirely variable. On one web site, its OAuth implementation may be secure, while on another, its implementation may be Swiss cheese. Even though a security consultant should generally look over any authorization implementation to ensure mistakes were not made, laymen can have a good understanding how reliable and secure their standardized authentication scheme is. A manager can ask their developers if some bullet points were adhered to, and then be reasonably confident in their security. Whereas with OAuth, the entire (complex) implementation needs to be reviewed from top to bottom by a top security professional to ensure it is secure. A manager has no adequate set of bullet points to discuss with their developers, as unique implementation details will drastically change the applicability of various points, with many important details still missing. At best, a manager can only get a false sense of security when OAuth is in use. The designers of OAuth respond to this point with a 71 page document of security issues that need to be dealt with!

What all this boils down to, is that OAuth is really just a set of ideas and recommendations on how to implement a unique authorization scheme. OAuth does not allow for interoperability as standards (usually) guarantee. Essentially, OAuth ensures that API authentication with your web site will be confusing, and will only work for the exact use-case for social networking sites described above.

Crippling Design - APUI

The common OAuth use-case described above is to allow for a user on one web site to allow their software to communicate for them with another web site. The workflow described above requires that a user navigate between web sites using their browser and manually enter their credentials and authorization to various aspects of their account as part of the overall software authorization process.

This means that OAuth (at least how it's normally implemented) only works between two web sites, with individual users, and that user intervention is required in order for software to authenticate with another web site. The fact that user intervention is required with manual input means that any API behind OAuth is not an API - Application Programming Interface, but an APUI - Application Programming User Interface, meaning user intervention is required for functionality. All in all, this cripples your API, or APUI as it should now properly be called.

Let us now focus on what cannot be properly done with OAuth in place:
  • Third party software which is not part of a web site cannot interact with an OAuth APUI.
  • One user in third party software cannot act on behalf of another user via an OAuth APUI.
  • Third party software cannot run automated processes on an OAuth APUI.
  • Organizations cannot ensure tight integration between various software and services they use.
Let us review an example case where a company launches a new online service for coordinating schedules and personal calendars between people. We'll call this new online service Calendar. The developers of Calendar create an APUI for it using OAuth, so third party software can integrate with Calendar.

As described above, a user needs to navigate between one web site and another in order to authorize software, with the two web sites sending each other information. What if one wants to integrate something which isn't a web site with Calendar? There's tons of desktop calendaring applications, Microsoft Outlook, Mozilla Lightning, Evolution, KOrganizer, Kontact, and more. If their developers want to integrate with the Calendar service, they need to embed a web browser in their applications now?

Furthermore, that may not even help, if the OAuth workflow requires information be sent to an existing web site, what web site is associated with a user's personal copy of software? Developers of that software are unlikely to create a web site with user accounts just for this integration. Even if they did, if the site goes down (temporarily), then the integration stops working. Even though, in theory, there should be no dependance of some extra web site between Calendar and the software trying to integrate with it.

Insecure

Also, as a security consideration, if an application does embed a web browser in it so it can authenticate with OAuth, the first requirement is no longer met - Untrusted software should not have access to a user's credentials. When using a standard web browser, once a user leaves the third party software's site and redirects to Calendar's site, the third party software cannot steal the information the user is entering. But when the third party software itself is the one which browses to Calendar's site, then it has access to everything the user is entering, including passwords.

Actually, this attack works on OAuth in every circumstance, including web site to web site, and I've actually used it in practice. A web site can embed a web browser via a Java Applet or similar, or have a web browser server side which presents the OAuth log in page to the user, but slightly modified to have all the data entered pass through the third party site. Therefore OAuth doesn't even fulfill its own primary security objective!

Incompatible with Enterprise

Next, once we go to the enterprise level, OAuth starts becoming much worse. Say a boss has his secretary manage his calendar for him, which is a very common scenario. In many OAuth setups, he cannot enter his Calendar credentials into the company-wide calendaring software running on a secure company server, which the secretary can then access. Rather, he would need to enter it directly in the browser the secretary uses, and stay there checking off many options. Also it is common that OAuth implementations are using security tokens which expire, meaning the boss will need to keep reentering his Calendar credentials again and again. Most bosses will just get fed up and give his secretary his credentials, especially if he's not physically near his secretary. It is also likely that the secretary will then write the password down. This gets compounded if multiple secretaries manage his schedule.

Now say an organization has software for which it would like to run background processes for all its users with Calendar. Perhaps every week it would like to automatically analyze the Calendar accounts of all department chiefs to find a good time for a weekly meeting. How would this work exactly? Every department chief would now have to go and enter their credentials into this software? Then do it again each time security tokens expire from past authentications?


Of course this is only the beginning. Since OAuth was designed for the likes of Twitter and Facebook which cater to individual personal accounts, the common implementations do not allow for hierarchical permissions as enterprise organizations need.

In enterprise infrastructure, software is already used which has well defined roles and access rights for ever single user. This infrastructure should be deciding who can do what, and to what level one user can act on behalf of another user. Once an organization purchases accounts for all their employees from Calendar, no employee should be able to turn off or limit what the enterprise management software can do with their Calendar account. The enterprise management software is the one who needs to make such decisions. This flies in the face of Untrusted software should not have full access to a user's account, but only limited access as defined by the user and A user should be able to revoke the permission they granted to allow particular untrusted software to work.

The amount of work involved to tightly integrate Calendar with existing infrastructure is also enormous from a user perspective. Every single account now has to have a user navigate across web pages and check all kinds of boxes until all the users are integrated. OAuth implementations generally forget to have administrator accounts which can do everything on behalf of other users in their organization.

It should be obvious at this point whether enterprise organizations would be willing to purchase accounts for their entire staff with Calendar, when integration into their existing infrastructure or new endeavors with the service will be difficult if not outright impossible. Imagine how much money Calendar sales now stand to lose from being unattractive to enterprise clients.

Repel third party developers

OAuth implementations also generally require that any software which uses their APUIs be preregistered in advance. This places extra burden on third party developers. The OAuth implementations also commonly require that the third party web site integrating be preregistered in advance, so you can say goodbye to your application being compatible with staging servers which may be at another URL. It also makes software much less attractive when one party sells software to another party, as every new client now needs to go through an application registration process for the URLs they plan to setup the software they are purchasing. Therefore third party developers are less likely to be interested in selling software which integrates with Calendar, as there's hassle involved with every single sale.

Recap

  • OAuth is not a standard, but a set of ideas, which does not allow third party developers to reuse knowledge, and places extra documentation burden on implementers.
  • OAuth as a whole has undefined security characteristics.
  • OAuth doesn't properly fulfill its primary security objectives.
  • OAuth doesn't work well outside social networking web site use-cases.
  • OAuth services are unattractive to enterprise organizations looking to integrate such services into their infrastructure.
  • OAuth services are less likely to have professional third parties sell software based upon them.


Solutions


If you're looking to implement authorization for your API, I recommend to sticking with well understood secure designs, such as HTTP Basic Authentication over SSL/TLS (or HTTP Digest Authentication).

In order to achieve a situation where users can securely authorize third party software, without giving over their personal credentials (passwords), I recommend that these services have a page where they can generate new credentials (keys) which the user can copy and paste. They can then name these keys themselves (avoiding application registration hassle), and set permissions upon them themselves. Since the user is the one initiating the key creation, and copying and pasting it themselves, they cannot fall prey to a man-in-the-middle attack where the third party software initiates the authorization process.

But remember the use-cases described here, and ensure that organizations have a way to access all user accounts company-wide, and without individual users being able to disable or limit that access.

Conclusion


If you want your service to be used by everyone out there, be well supported by third parties, and to have them create all kinds of interesting software with it, do not use OAuth.

Even the original social networking sites behind OAuth decided they really need other options for different use-cases, such as Twitter's xAuth, or Yahoo offering Direct OAuth, which turns the entire scheme into a more complicated version of HTTP Basic Authentication, with no added benefits. Perhaps the most damaging point against OAuth, is that the original designer behind it decided to remove his name from the specification, and is washing his hands clean of it.

I find it really amazing at how blind many big players are these days to all the problems with OAuth. When I first heard that IBM, a major enterprise player started offering services only accessible with typical utterly crippled OAuth, I was overcome with disbelief. Yet at the same time, I hear that they're wondering why they're not seeing the sales they used to with other services they offer, or compared to the competition.

I'm even more amazed to see other big companies throwing away their currently working authentication systems for OAuth. Followed by them wondering why many third party developers are not upgrading to support the new authentication scheme, and clients jumping ship to inferior services.

It seems many developers and managers out there simply don't get it. If you know anyone like that, show them this article.

24 comments:

henke37 said...

I had no idea things were this bad. It's all solvable, but will it be solved? And more importantly, will it be solved correctly?

Joe P. said...

I was recently brought in for the preliminaries on a consulting job to "improve API security" in a certain company which has software targeted specifically for the enterprise market. The sort of software which only makes sense when coordinating the information of at least a dozen employees and would be meaningless on a user-by-user basis.

As you might guess, when I got there, I was told proudly by one of their project managers, like a 3 year old excitedly showing you the crayon smearings he made on your nice clean living room wall, that the company had decided to switch their API authentication to OAuth and phase out all other authentication models. My first response was to ask him why they were engaging in the mutually exclusive activities of bringing in extra help to improve security and switching to OAuth.

When he didn't get what I was implying, I asked directly why they were switching to OAuth when their previous authentication methods were completely secure (or at least, as secure as this type of company ever has the combined brainpower to manage). His response was to foolishly say "Don't you know? OAuth is even more secure. It's the next generation of secure."

I then politely asked if he could explain to me what areas of security OAuth covers where traditional authentication schemes have holes or vulnerable points of attack. Naturally, he failed to even understand the need for that question, much less respond with anything intelligent. Needless to say, I didn't take the job, and the simpering idiot probably still thinks that OAuth is the Terry McGinnis to HTTP Digest's Bruce Wayne. (I liked Bruce better anyway.)

All this teaches me two things:
1. OAuth is the new buzzword which has all the hype with none of the function.
2. A mauve OAuth scheme has the most RAM.

It seems that OAuth was created for a very specific use-case, one which it excels at filling, and nothing outside of that use-case. I can't imagine anyone who really knows security could possibly defend the use of OAuth for anything other than the case you mentioned. Unfortunately, the decisions are, as always, made not by those who understand but by those who think they understand. We live in a world of limited intellect where a catchy name and good PR do more than a solid foundation and hard facts ever accomplish. Managers love using reinvented wheels with catchy names ("ORoller"s, "SpinZoom"s, "Weelz 2000"), and security, being an area in which many claim to be knowlegdable while few are, is a prime target for reinvention.

Therefore, I think your "Solutions" section is lacking. What we really need to do is rename traditional HTTP-level authentication to "PassGird", convince everyone it's a new thing, and get all the morons to start whispering about it.

computerfreak1k said...

Great article, I've always known the same thing, thanks for putting it into words. Maybe now we'll stop seeing all these crippled APIs

insane coder said...

For even more information, you can see this great article on why it's not possible to properly offer desktop or open source applications with OAuth.

There's also another discussion here about many of its flaws.

Bruce Gold said...

This is a very important article with tremendous repercussions. Why are you not publishing this on Slashdot?

insane coder said...

Hi Bruce.

Anyone can submit a story to Slashdot. If you'd like to, go ahead.

I have gotten some good feedback here as well as over IRC, so I guess this article is Slashdot worthy. I'll go see about writing a submission for Slashdot.

Matt Ray said...

IC - thanks for a great article. I really appreciate the angles you approached it with. As someone who has worked with OAuth for social media connections, I completely agree with your comments.

Also, I am glad this made it to the front page of /.

Jeremy Bush said...

I think it's important to note that this is talking baout oauth2, and not oauth1, which is pretty different.

peter said...

Your stylesheet is illegible. Would you kindly consider updating it to something other than green on green? Thanks.

dclawrence said...

@JeremyBush OAuth 1 unfortunately suffers the same problems as mentioned in the article. It has a slightly different implementation, but the problems with authenticating 3rd party apps and requiring user input are identical.

OAuth2 was (in part) meant to solve the problems with authenticating things other than websites and it didn't, by a long way.

Michael Thomas said...

You might be interested that I brought up the problems with native app clients on the ietf working group several years ago, and got snarled at by many including Hammer at the time.

Here's a link to the working group's archives: http://www.ietf.org/mail-archive/web/oauth/current/msg07411.html

Corley Kinnane said...

I think it's a pity that the original security specs like Basic and Digest authentication are not taken more seriously, but it actually just represents a flaw in the original spec similar to why CSS came along later... for people who don't understand the security implications of re-implementing security in HTML without the browsers help, its the look and feel of the secure mechanism that caused them to change in the first place.
All we really need is an extension to authentication that allows for the broswer embedding the required fields into the look and feel provided by the site - then a simple browser verification of the fileds that is availbale to the user just like the SSL symbol and certficiate info is available.
No longer a standard popup for credentials, but CSS laden with the fields being verified as produced by the browser.

Milandro said...

I have to agree with @JeremeyBush a bit in that a lot of the grievances in this article really target the OAuth 2 framework instead of OAuth 1.0a protocol.

I have seen and have even written an OAuth 1.0a client-side protocol implementation that authenticates to every service I have tested against so far without service specific exceptions (twitter, ubuntu one, flickr, dropbox). It seems like OAuth 2 describes the framework type behavior which is inconsistent between services vs the OAuth 1.0a protocol behavior which is pretty consistent between services.

As far as I have seen it is also not really necessary to bundle a web browser with an application for stand-alone app development when using OAuth as long as you are capable of opening a web browser to a specific URL and have the permissions to service a port as localhost on the machine (which is definitely a mobile problem but not so much desktop). There are inherit security problems with bundling token secrets in an application but that does not seem to be the target of the article.

OAuth 1.0a tokens do not expire unless the user or services specifically cause them to expire. From what I understand OAuth 2 tokens expire but can be refreshed without user input although the flexibility of OAuth 2 probably makes this service specific.

While there are problems with implementing and scaling OAuth 1.0a at the enterprise level with session management, etc. I think it was the attempt to better tailor the protocol for enterprise usage that left the OAuth 2 standard in the state it is in. I cannot say I have extensive experience with enterprise level usage of OAuth and that is mostly from reading of Hammer's Road to Hell article

I also agree with the idea behind Joe p.'s reply. OAuth is a tool for specific type of job, gaining access to a user's resources hosted in a separate service in a manner that does not intrude on their credentials. I think OAuth is pretty good at it's job and OAuth 1.0a is much more consistent for the 3rd party app developer while OAuth 2 probably helps integrate better with the resource provider at the expense of breaking consistency for the 3rd party app developer.

Michael said...

Yeah things are getting pretty bad. It's like a cargo cult out there with everyone just imitating a few visible players in the hopes that it will all work out just right. This is being done by the same developers that reinvented inscrutable tightly-coupled spaghetti code in object-oriented languages, that reinvent message queues on every datastore under the sun, that switch from SQL datastores to NoSQL without building the required safety nets in to their apps, and so on.

Long live Uncle Bob and Kirk Knoernschild! Long live RabbitMQ and OSGI.

Kris Reeves said...

@Michael

All I gather from the mailing list is that you don't know how to debate. I keep clicking responses and being baffled that you're STILL going on and saying the same thing over and over again. But the way you're saying it doesn't really address the point you are trying to make, so it's no wonder nobody agreed with you.

If you want to be effective, you should involve yourself as you were invited to do: propose changes rather than demand work from everyone else. The best way to get something done is to do it yourself.

Antonio Sanso said...

While I may agree with some of your comments I think you failed to offer a valid alternative to OAuth

Michael Thomas said...

@Kris: perhaps you're not familiar with IETF. When the editor who repeatedly dismisses the entire problem as irrelevant says to "propose text", he's really telling you to fuck off in a different way. Many people including working group chairs were rather taken aback by the working group participants' bad manners -- especially since I a) have no stake in its outcome and b) came questioning whether what I was seeing was actually the case (and was invited by the working group chair to bring it up after a private conversation). And in fact, I suggested many concrete changes to the threats draft. My solace is that OAuth is a disaster so that the likelihood of it being widely deployed beyond the usual suspects is low.

insane coder said...

Hello Antonio Sanso,

I'm not looking to replace OAuth, at least not in the cases it was best designed for. But I am looking for it to stop being used as authentication for APIs.

What aspects of my suggestions did you find lacking?


Hello Michael Thomas,

The reason why I wrote this article is because many big players are switching to using OAuth for their APIs, and it is becoming a disaster.

IBM, Microsoft, Cisco, and others are creating new services with APIs behind OAuth, or switching existing authentication routines to OAuth.

Their services are ending up only working with web browsers, and in very limited annoying scenarios. This makes designing desktop software or advanced software which makes use of these services very difficult, extremely user unfriendly, or just plain impossible.

I've spoke to some managers and developers at respective corporations, and no one can give a clear reason as to why they're using/switching to OAuth, but all claiming they have to use it. They're also looking for more third party adoption, but not getting it and wondering what's wrong, yet don't want to listen to feedback.

John Thomas said...

It may be that
1. inter-website communication
2. enterprise software
3. native app-clients
have different needs, and OAuth was a protocol designed for #1 and then hammered into #2 and #3. Maybe there is a solution for all three, but it may need a careful consideration of what each use-case needs.

For #3, the login via browser workflow especially just bugs me. I hate the idea that one application has to open up another application just for a login. If we are going outside the application for the login, let's make an easier api for websites to access OS-level authentication.

My personal use case is I want to make a nice tool to pull data out of my social networking sites and use it for my custom applications without ever logging back into the website if I do not want to. Essentially what I want is something like IMAP or POP for data. Oauth does not match that need.

Oh well, back to scraping websites and parsing out the data.

Antonio Sanso said...

> But I am looking for it to stop being used as authentication for APIs.

OAuth has been designed (as you know) as an authorization protocol.

Should you look toward authentication you better look at Open ID Connect

Antonio Sanso said...

>What aspects of my suggestions did you find lacking?

there is a bit too much FUD for my taste

Marta said...

I agree with Antonio.
If you prefer using Basic Authentication, going to a website to copy&paste codes...what is the advantage for the user or developer?
Then you might as well use OAuth2 resource owner grant type and at least get some benefit by not having to store username/password locally and reauthenticate every time.

insane coder said...

Hello Marta:

If you manually initiate the process and copy and paste, you avoid man-in-the-middle attacks.
The attack descrived above can only occur where the website initiates the process, as is common in OAuth workflows.

The advantage of Basic Authentication for developers is that it's already well supported by major libraries, and standardized. With OAuth, they can be using 1, 1a, dozens of flavors of 2, not something that your favorite libraries is necessarily going to support.

Note point 6 here: "Document your OAuth implementation".

It is quite sad when there's a supposed "standard" that has many implementation details that need documenting to use.

Standards are supposed to be about interoperability.

insane coder said...

Oh and in case it wasn't clear, you don't need to store actual user credentials and use those via Basic Auth.

You store your special API keys and send those via Basic.