Re: HTTP 401 and explanation when trying to read a private topic

Browsers and search engines do. Although for a search engine a 401 and 404 probably isn't very different.

Ben
SVN repository for my extensions - The thread
Quickmarks 0.5
“Question: How does a large software project get to be one year late? Answer: One day at a time!” - Fred Brooks

Re: HTTP 401 and explanation when trying to read a private topic

Connor wrote:

How many users know what a 401 error means?

The original idea was to send a 401 code and a message explaining the user doesn't have permission to view the topic (or to send a 404 code and a message explaining the link doesn't exist). I assumed you meant that, since changing it to send a 401 code though leaving the message vague would just be fairly pointless...

Re: HTTP 401 and explanation when trying to read a private topic

My point is, changing the message solves the problem, whether we send a 404 or a 401 is pretty much irrelevant to most users, and sending a 404 is not incorrect as far as I can tell, but sending a 401 is bad security practise, and gives no benefit as far as I can see.

Re: HTTP 401 and explanation when trying to read a private topic

From reading the RFC, it would seem that 401 isn't suited for the job, since FluxBB doesn't allow HTTP Auth:

RFC 2616 wrote:

401 Unauthorized


   The request requires user authentication. The response MUST include a
   WWW-Authenticate header field (section 14.47) containing a challenge
   applicable to the requested resource. The client MAY repeat the
   request with a suitable Authorization header field (section 14.8). If
   the request already included Authorization credentials, then the 401
   response indicates that authorization has been refused for those
   credentials. If the 401 response contains the same challenge as the
   prior response, and the user agent has already attempted
   authentication at least once, then the user SHOULD be presented the
   entity that was given in the response, since that entity might
   include relevant diagnostic information. HTTP access authentication
   is explained in "HTTP Authentication: Basic and Digest Access
   Authentication" [43].

Thus a 404 would be more suited:

404 Not Found


   The server has not found anything matching the Request-URI. No
   indication is given of whether the condition is temporary or
   permanent. The 410 (Gone) status code SHOULD be used if the server
   knows, through some internally configurable mechanism, that an old
   resource is permanently unavailable and has no forwarding address.
   This status code is commonly used when the server does not wish to
   reveal exactly why the request has been refused, or when no other
   response is applicable.

And yes, I do agree that giving a slightly more descriptive message won't impact security too much.

Ben
SVN repository for my extensions - The thread
Quickmarks 0.5
“Question: How does a large software project get to be one year late? Answer: One day at a time!” - Fred Brooks

Re: HTTP 401 and explanation when trying to read a private topic

Connor wrote:

My point is, changing the message solves the problem, whether we send a 404 or a 401 is pretty much irrelevant to most users, and sending a 404 is not incorrect as far as I can tell, but sending a 401 is bad security practise, and gives no benefit as far as I can see.

I thought the problem was that disclosing the fact the post exists is bad security practice (whether it be by sending a 401 or displaying a message saying so). I don't see how disclosing the fact a post exists by sending a 401 is any worse security wise than sending an easy to read message saying the same thing.

I am not saying sending a 401 is appropriate, just that I don't see how sending different messages but the same code solves the security issue Smartys brought up. (Also personally I don't think it is much of a security issue, where's the harm in knowing it exists?)

Re: HTTP 401 and explanation when trying to read a private topic

You can give an error message without impacting security by just giving a general "this might be a page, you can try logging in" or whatever.

Re: HTTP 401 and explanation when trying to read a private topic

Connor wrote:

My point is, changing the message solves the problem, whether we send a 404 or a 401 is pretty much irrelevant to most users, and sending a 404 is not incorrect as far as I can tell, but sending a 401 is bad security practise, and gives no benefit as far as I can see.

First, I should have re-read the RFC, and not Wikipedia summary. Yes, 401 isn't relevant because of the http auth, 403 is the right one.

Next:

HTTP error are mostly not for humans, but for machines: search engine's spider, but it could be wget, a Firefox extension, whatever; we don't know all the current tools available, and we certainly don't know what the future holds. That's the point of sticking to protocol, it allow other people to do smart thing with that protocol we didn't think about in the first place.

The RFC got it right to the point:

If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.

We are back to the issue of: does FluxBB doesn't handle all errors the same, does it acknowledge that an unknown URL isn't the same as one the user isn't allow to see.

I do think it's important that Flux doesn't handle all error the same. When the average user follow a link and is sent an error in the face, the software should explain the error the best way it can, and help the user correct it. Basically, it should be user friendly.

And as already pointed, if Flux explain in plain English that X is a private topic the user isn't allowed to read at the present time, and that Y is a URL it doesn't understand and maybe the user should use the Search engine to find whatever he was looking for (and it both should); then I don't understand what possible reason could there be for not sending the appropriate HTTP error code header so that machines can understand the same thing.

Although, I agree the most important point (and by far) is the plain English explanation.

Apart from the bug/feature itself, I'm still curious about the security issues people talk about. I'm sorry to be thick, but I can't imagine what the security related topic might be in this area.

Re: HTTP 401 and explanation when trying to read a private topic

No one is saying that we should return a 404 but say the user doesn't have permission, afaik the solution is to say that the page may exist, and the user may not have permission to view it. This is quite different to simply saying they don't have permission.

If we return a 404 with a message that does not confirm if pages exist or not I don't see a problem.

As for the security concern, Smartys explained and linked to similar problems already.

Re: HTTP 401 and explanation when trying to read a private topic

Connor wrote:

No one is saying that we should return a 404 but say the user doesn't have permission, afaik the solution is to say that the page may exist, and the user may not have permission to view it. This is quite different to simply saying they don't have permission.
If we return a 404 with a message that does not confirm if pages exist or not I don't see a problem.

I see two, at the first glance:

- it doesn't help the user, at all. I browse through acme.com and follow a link explaining why Acme Gizmo is the best alive. If all I get is an error message saying: “ok the page you asked, well, I don't have it; or maybe I have it but I won't give it to you; or maybe my database is down; or maybe I've been taken over by aliens and they're after you” I won't bother with anything. I'll close the tab, I'll close the original Acme tab, and will move to the next point of order of my day.
It would be marginally better than the “thing” we currently have, but it's still confusing the user on purpose. We have the information, why not explain it to the user?

- for a less average web user, I think they may (and we can't blame them) take it as a personal insult. They know the server has the damn data, they can easily check if the topic is either private or has existed and being deleted, or if the link if bogus altogether. Why make the user life more difficult on purpose?

As for the security concern, Smartys explained and linked to similar problems already.

Erh... unless I'm blind, no he didn't.

Smartys wrote:

No. You can't prove that the topic exists, which is the important part. It may have been deleted, it may not have been created yet, or it may be private, but you can not say with 100% certainty which it is. The idea is not specific to topics and forums but is important in general in order to prevent information from leaking.

And none of his links pointed to a similar issue, as far as I can see.

It's all relative to authentication. The general idea is, login is half the auth, if you give away the existence of a login, you're screw. I mostly disagree with this (the whole user list of any FluxBB forum is public, and security through obfuscation slightly delay the break in, but significantly delay the security solving) but that's not even the issue here.

We're not asking (yet tongue ) about the auth, we're asking the software to qualify the server errors, in other words to disclose the existence of _content_.

What's the security issue with content? What can someone possibly do when he knows 100% a specific content exist? Especially when he knew for 99.9% certainty before that?

To take a sentence off Smartys quote, “The idea is not specific to topics and forums but is important in general in order to prevent information from leaking”: but as admin we want to propagate information! That's the whole point of a forum software, isn't it?

Again, what would be a scenario where a hacker can do something evil with knowing 100% that the topic n is private? I just don't understand that.

Re: HTTP 401 and explanation when trying to read a private topic

I don't see how saying "Bad Request. This page may exist but you do not have permission to view it" is any different to saying "You do not have permission to view this page" from the users perspective, except the first one does not imply the page exists. We can't be sure the user will be able to access the page after logging in anyway.

The security point is not really that its a problem if a user knows if a topic id exists, the point is that you shouldn't disclose any information that the user does not have permission to know. This is the same as not telling someone if the username they tried to login as exists or not. I don't think this is security by obscurity or obfuscation, we aren't hiding something, we're just not giving out information the user doesn't have permission to have. Just because you can't imagine a situation where this is a problem, doesn't mean its fine to allow people to access that information.

but as admin we want to propagate information! That's the whole point of a forum software, isn't it?

No, its not the whole point, otherwise we wouldn't have permissions would we?

Re: HTTP 401 and explanation when trying to read a private topic

Connor wrote:

I don't see how saying "Bad Request. This page may exist but you do not have permission to view it" is any different to saying "You do not have permission to view this page" from the users perspective, except the first one does not imply the page exists.

Except the two first word, that's probably fine _if_ there's another message for a “real” 404.

To take a practical example: if this is what you want to return when we try to access http://fluxbb.org/forums/topic/1991/, and you'll return “sorry this URL doesn't exist” (or something) for http://fluxbb.org/forums/topic/acme/ that's ok.

The “may” in your sentence is strange, but whatever.

We can't be sure the user will be able to access the page after logging in anyway.

One thing at a time, that was next in my pipe smile

The security point is not really that its a problem if a user knows if a topic id exists, the point is that you shouldn't disclose any information that the user does not have permission to know. This is the same as not telling someone if the username they tried to login as exists or not.

I don't think it's the same at all (admitting that the auth thing is right, which it's not here, but let's take it one thing at a time tongue ).

I understand the generalization, the global idea of “you shouldn't know that” generalized to everything. It's a nice theorem... in theory. But it's wrong, and private topic is a great counter example.

You say there's no issue, security or otherwise, letting the user know a topic is private. So you apply a global theoretical theorem to your decision making, and ending up confusing the average user for no reasons.

You don't see the benefit for the end user to have a distinct error message for an unknown URL and a private topic?

we're just not giving out information the user doesn't have permission to have.

Can I rephrase my bug fix (or feature modification, or whatever) to: give the user permission to have this, then? smile

Just because you can't imagine a situation where this is a problem, doesn't mean its fine to allow people to access that information.

My question was more: Can you? Can any of the core dev? Can anyone on the web? At all?

Again I'm talking about this very specific subject: disclosing the private versus "doesn't exist" status of a FluxBB URL.

By the way, if I try to get into Fluxbb.org admin panel, I get: “You do not have permission to access this page.” Why here I got a clear error message, and not for a private topic?

Re: HTTP 401 and explanation when trying to read a private topic

Jérémie wrote:
Connor wrote:

I don't see how saying "Bad Request. This page may exist but you do not have permission to view it" is any different to saying "You do not have permission to view this page" from the users perspective, except the first one does not imply the page exists.

Except the two first word, that's probably fine _if_ there's another message for a “real” 404.

To take a practical example: if this is what you want to return when we try to access http://fluxbb.org/forums/topic/1991/, and you'll return “sorry this URL doesn't exist” (or something) for http://fluxbb.org/forums/topic/acme/ that's ok.

No the point is there is 1 message, otherwise the user would know if the topic exists or not.


Jérémie wrote:

(admitting that the auth thing is right, which it's not here, but let's take it one thing at a time tongue ).

It was an analogy, to help you understand, not a discussion point

Jérémie wrote:

I understand the generalization, the global idea of “you shouldn't know that” generalized to everything. It's a nice theorem... in theory. But it's wrong, and private topic is a great counter example.

No, you think its wrong, very different

Jérémie wrote:

You say there's no issue, security or otherwise, letting the user know a topic is private. So you apply a global theoretical theorem to your decision making, and ending up confusing the average user for no reasons.

I don't think the average user is confused, especially when we can't really tell them if they could have access to the page anyway

Jérémie wrote:

You don't see the benefit for the end user to have a distinct error message for an unknown URL and a private topic?

I see little benefit, and not the point at all

Jérémie wrote:

we're just not giving out information the user doesn't have permission to have.

Can I rephrase my bug fix (or feature modification, or whatever) to: give the user permission to have this, then? smile

If you're going to argue like this, I'm going to stop arguing

Jérémie wrote:
Connor wrote:

I don't see how saying "Bad Request. This page may exist but you do not have permission to view it" is any different to saying "You do not have permission to view this page" from the users perspective, except the first one does not imply the page exists.

Except the two first word, that's probably fine _if_ there's another message for a “real” 404.

To take a practical example: if this is what you want to return when we try to access http://fluxbb.org/forums/topic/1991/, and you'll return “sorry this URL doesn't exist” (or something) for http://fluxbb.org/forums/topic/acme/ that's ok.

No the point is there is 1 message, otherwise the user would know if the topic exists or not.


Jérémie wrote:

(admitting that the auth thing is right, which it's not here, but let's take it one thing at a time tongue ).

It was an analogy, to help you understand, not a discussion point

Jérémie wrote:

I understand the generalization, the global idea of “you shouldn't know that” generalized to everything. It's a nice theorem... in theory. But it's wrong, and private topic is a great counter example.

No, you think its wrong, very different

Jérémie wrote:

You say there's no issue, security or otherwise, letting the user know a topic is private. So you apply a global theoretical theorem to your decision making, and ending up confusing the average user for no reasons.

I don't think the average user is confused, especially when we can't really tell them if they could have access to the page anyway

Jérémie wrote:

You don't see the benefit for the end user to have a distinct error message for an unknown URL and a private topic?

I see little benefit, and not the point at all

Jérémie wrote:

Just because you can't imagine a situation where this is a problem, doesn't mean its fine to allow people to access that information.

My question was more: Can you? Can any of the core dev? Can anyone on the web? At all?

That isn't the point.

Jérémie wrote:

By the way, if I try to get into Fluxbb.org admin panel, I get: “You do not have permission to access this page.” Why here I got a clear error message, and not for a private topic?

Because that would be security by obscurity? Anyone can download FluxBB and find out the URL of the admin panel.