[SOLVED] tinyproxy filtering http headers unexpectedly

[wwdev16]

tinyproxy appears to be filtering non-standard http headers even though I believe
it is configured to not do that. This causes problems with the new version of proton mail.

Not finding anything useful on https://github.com/tinyproxy/tinyproxy/issues or
in man tinyproxy or man tinyproxy.conf.

In /var/log/tinyproxy/tinyproxy.log I see:

Code: Select all

Not sending client headers to remote machine

All of thes commands return nothing:

Code: Select all

grep -v '#' /etc/tinproxy/tinyproxy.conf | grep -i Anon
grep -v '#' /etc/tinproxy/tinyproxy.conf | grep -i Via
grep -v '#' /etc/tinproxy/tinyproxy.conf | grep -i Host
grep -v '#' /etc/tinproxy/tinyproxy.conf | grep -i XTinyproxy

Is there some config directive needed for web clients to have non-standard headers sent?

$ qlist -ICv tinyproxy:

Code: Select all

acct-group/tinyproxy-0                                                                                          
acct-user/tinyproxy-0                                                                                           
net-proxy/tinyproxy-1.11.0_rc1

$ equery u tinyproxy:

Code: Select all

 * Found these USE flags for net-proxy/tinyproxy-1.11.0_rc1:
 U I
 - - debug             : Enable extra debug codepaths, like asserts and extra output. If you want to get
                         meaningful backtraces see
                         https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
 + + filter-proxy      : Enable filtering of domains/URLS 
 - - reverse-proxy     : Enable reverse proxying 
 - - test              : Enable dependencies and/or preparations necessary to run tests (usually controlled by
                         FEATURES=test but can be toggled independently)
 - - transparent-proxy : Enable transparent proxying 
 - - upstream-proxy    : Enable upstream proxying 
 - - xtinyproxy-header : Include the X-Tinyproxy header

[user]

A wonderful world of open source.

Code: Select all

# ebuild /var/db/repos/gentoo/net-proxy/tinyproxy/tinyproxy-1.11.0.ebuild prepare

# grep -r "Not sending client headers to remote machine" /tmp/portage/net-proxy/tinyproxy-1.11.0/work/tinyproxy-1.11.0/
/tmp/portage/net-proxy/tinyproxy-1.11.0/work/tinyproxy-1.11.0/src/reqs.c:                             "Not sending client headers to remote machine");

# grep -B8 "Not sending client headers to remote machine" /tmp/portage/net-proxy/tinyproxy-1.11.0/work/tinyproxy-1.11.0/src/reqs.c
        /*
         * Don't send headers if there's already an error, if the request was
         * a stats request, or if this was a CONNECT method (unless upstream
         * http proxy is in use.)
         */
        if (connptr->server_fd == -1 || connptr->show_stats
            || (connptr->connect_method && ! UPSTREAM_IS_HTTP(connptr))) {
                log_message (LOG_INFO,
                             "Not sending client headers to remote machine");

Ready to verify possible reasons?

[szatox]

Is there some config directive needed for web clients to have non-standard headers sent?

It does allow you to control what headers will be passed to the server.
AFAIR anonymous mode strips all non-essential headers.

But.... if I were you, I'd start with starting tcpdump on the machine with your proxy. And trying to connect anywhere. Possibly over HTTP, so I could easily eavesdrop on that conversation, and perhaps see how it fails.

[wwdev16]

I'm going to call it a bug. Using privoxy instead doesn't have the problem.

From what I know of the HTTP spec, the CONNECT method doesn't require
stripping http headers from the initial connection to the destination. So I
suspect tinyproxy's behavior as shown by user's code snippet isn't correct.

Thanks for the help.