0

I am trying to connect to a supplier API but they have a whitelist of the IPs which can consume their API. I gave them my server IP so that they could add it to the whitelist but it is still not working.

I suspect this is coming from my company proxy which hides IP, so I did some tests : weirdly enough, I noticed that the real IP of my server could not be detected on most websites like "who.is" or "whatismyipaddress.com". But as soon as I would try the same thing on websites using HTTP such as "http://www.mon-ip.com/en/my-ip/", then the proxy is detected and my server IP is found out.

I don't have a very deep understanding of HTTP and HTTPS so I have no idea how using http makes it possible to detect the proxy and find out the real IP. I tried checking request headers and it doesn't seem that the IP is transmitted in HTTP requests...

I've looked up some topics on SO and couldn't manage to find anything closely related to that matter (best I found was How can a website find my real IP address while I'm behind a proxy? which is somehow confusing me even more). Would someone be able to enlighten me on how http makes it possible to detect proxies and find out the real IP being used ?

Henry Mont
  • 111
  • 4
  • IIUC correctly what you try to achive, you don't wants to hide your IP address at all but the opposite, don't you? – TorstenS Jun 27 '22 at 15:27
  • Yes, that's what I'm trying to achieve. And trying to do so, I stumbled upon that weird behavior with HTTP and HTTPS that I am now trying to understand. But yes, I should probably edit the question title. – Henry Mont Jun 28 '22 at 09:29

3 Answers3

1

Whitelists on public HTTP(S) based APIs are usually layer 2, i.e. they care for the (public) IP address(es) which is trying to make a connection. I would have a hard time believing they would care for anything which is in an X-Forwarded-For header, as this would be extremely easy to spoof and provide little extra security.

So you will have to provide them the public IP of your company's proxy server to have this whitelisted, not the internal IP of your server. (I assume it's an internal one.) Please note, it may be the case that your company is using some battery of proxies or different Internet breakouts, so you need to be sure you have all potential public IP addresses whitelisted.

TorstenS
  • 818
  • 4
  • 9
  • Yes I am actually trying to give them the range of IPs used by our proxies, but it's very troublesome because they want those IPs to be owned by our company. The problem is we are using Zscaler. I'm starting to think the only way is to ask IT to create a routing out of zscaler for those API requests... – Henry Mont Jun 28 '22 at 06:33
1

Ok I think that I have found the answer to my question. It all comes down to my company using Zscaler.

It is said in the doc that :

ZIA acts as an inline proxy: Zscaler terminates the original connection from the customer’s device or network and initiates a new, direct connection to the destination content server on behalf of the user. The source IP address seen by the content server is a public-egress IP address from the Zscaler data center, and not the original IP address of the enterprise user’s device.

And it is also said that :

XFF (x-forwarded-for) is inserted by default for all HTTP traffic going through Zscaler. If the destination application or content server can read and interpret the incoming XFF header, it can apply its source IP-address-based application access rules.

So I still don't know why I am not seeing the XFF header when checking my requests, but that's the most plausible explaination.

Manjula
  • 180
  • 6
Henry Mont
  • 111
  • 4
0

I don't have a very deep understanding of HTTP and HTTPS so I have no idea how using http makes it possible to detect the proxy and find out the real IP. I tried checking request headers and it doesn't seem that the IP is transmitted in HTTP requests...

When using HTTP, the proxy can inject the X-Forwarded-For header with the IP of the originator.

When you use HTTPS (HTTP+TLS), the proxy can't access the headers - as they are encrypted, and is thus unable to add the X-Forwarded-For headers.

vidarlo
  • 14,890
  • 2
  • 43
  • 56
  • So without any access to proxy's logs, I can't see what headers are injected ? – Henry Mont Jun 27 '22 at 07:56
  • 1
    That would normally not be logged. But the configuration will tell you. You can also set up a website that [prints the headers](https://www.php.net/manual/en/function.getallheaders.php). I made a [quick one](http://etherkiller.org/headers.html) you can check with. – vidarlo Jun 27 '22 at 08:10
  • Well that is weird, I don't see any X-Forwarded-For header. It doesn't seem like the real IP is being transmitted that way... – Henry Mont Jun 27 '22 at 08:30
  • It's not if you access it over https as the proxy can't inject it. – vidarlo Jun 27 '22 at 08:42
  • I tried accessing it over http and there isn't any X-Forwarded-For header. That would mean the proxy isn't injecting my real IP. So I still don't know how http://www.mon-ip.com/en/my-ip/ is finding my real IP. – Henry Mont Jun 27 '22 at 08:55
  • 1
    your server configured to use a proxy? it could be one of these that performs your https lookups for you on your behalf, then inspects the data, then re-encrypts and sends you the response? if yes, could that proxy be ignoring your http request because there's no work needed to inspect the traffic? (check the cert in your https response: look at the root cert) – brynk Jun 27 '22 at 12:19
  • 2
    A traditional 'CONNECT' proxy can't see or modify the content of HTTPS, but many proxies today are 'intercepting' proxies that decrypt and re-encrypt HTTPS, mostly for various kinds of security monitoring like IDS/IPS or DLP; these _could_ add a header, but I don't know if they do. In any case there's no reason for the server to _trust_ a client-side proxy's header. – dave_thompson_085 Jun 28 '22 at 01:36
  • @brynk I doubt that IT would install a proxy that let the http requests go through as they please ahah but who knows. So I should check who signed the certificate to see if the proxy has tampered with the http request then ? I'm sorry but I have no idea how to do that, didn't find much online, is it supposed to be in the header ? – Henry Mont Jun 28 '22 at 09:43
  • @dave_thompson_085 Yes, it is actually inserting a XFF header, thanks :) But it is indeed useless to me cuz the API won't trust that... – Henry Mont Jun 28 '22 at 09:57
  • 1
    @henry-mont not being flippant .. these proxies actually perform the request on your behalf as a *client*, then re-encrypt the data and send it to you to satisfy your query, encrypting the response as if they were the *server* - to achieve this, they need to instal a root cert into your local host trusted store - when you inspect the api host's certificate in your browser it will be issued by the proxy and signed by the root (not the api server!) .. see: https://security.stackexchange.com/a/133261 and https://www.thesslstore.com/blog/ssl-inspection/ .. ps. the proxy won't bother the http req. – brynk Jun 28 '22 at 12:03
  • @brynk Yes that's what I ended up finding out from my research, I guess this is more secure for the company... But that means I will have to ask IT to set up a routing around the proxy, this will troublesome :'( – Henry Mont Jun 28 '22 at 12:48
  • 1
    if the external ip-addy of the proxy is fixed or in a predictable range, you can have your api provider allow this (instead of your server's ip) - another idea to try may be to use some cloud service-provider endpoint: in my experience these are usually direct (eg. aws-gateway) .. use `ctrl+shift+i`/ `f12`/ devtools in your browser and connect to a s3 bucket (or equiv.) and you'll be able to tell the difference between that and an intercepted tx – brynk Jun 28 '22 at 18:01
  • @brynk Well we are using zscaler, and the range of IPs that can be used is huge (the netmask would be 255.255.0.0). And even if I could predict a smaller range, my API provider want the IPs to be owned by us... I'm afraid I will have the same problem with a cloud service provider. I'll try it if nothing else is working out. I will first put some pressure on IT and the API Provider. – Henry Mont Jun 29 '22 at 12:26