We recently encountered a situation with a Starman --> Apache --> NGINX setup where responses from /api/v1/patrons (particularly when doing pagination) could fail with an NGINX error message like this: upstream sent too big header while reading response header from upstream We determined that on the proxy server in question, the default NGINX proxy_buffer_size value of 4096 (the PAGESIZE on that server) was a bit too small for the Link header returned by the API response. Consequently, example NGINX configs in the code and the wiki should include the following: proxy_buffer_size 8k; This is related to bug 31104, but its fix of splitting up the Link headers won't be sufficient, as NGINX will drop the response if the total size of the headers is greater than proxy_buffer_size.
Created attachment 145635 [details] [review] Bug 32716: larger proxy_buffer_size in NGINX example config This patch updates the example NGINX config to increase the proxy_buffer_size to 16k. The default value of 4k (on some platforms) has empirically been shown to be a bit too small for the Link headers emitted by the REST API when pagination is requested. To test ------- [1] Set up a Koha system with NGINX as a reverse proxy in front of it (either in front of Apache or in front of of Starman). [2] Perform a patron search that returns at least two pages of results and navigate to the second page. [3] Note that the navigation can fail with a 502 HTTP error and an "upstream sent too big header while reading response header from upstream" error in the NGINX log. The problem is most likely when the pagesize of the server running NGINX is 4096 bytes. [4] Update the NGINX configuration per this patch and restart NGINX. [5] This time, repeating step 2 should work. Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Created attachment 148911 [details] [review] Bug 32716: larger proxy_buffer_size in NGINX example config This patch updates the example NGINX config to increase the proxy_buffer_size to 16k. The default value of 4k (on some platforms) has empirically been shown to be a bit too small for the Link headers emitted by the REST API when pagination is requested. To test ------- [1] Set up a Koha system with NGINX as a reverse proxy in front of it (either in front of Apache or in front of of Starman). [2] Perform a patron search that returns at least two pages of results and navigate to the second page. [3] Note that the navigation can fail with a 502 HTTP error and an "upstream sent too big header while reading response header from upstream" error in the NGINX log. The problem is most likely when the pagesize of the server running NGINX is 4096 bytes. [4] Update the NGINX configuration per this patch and restart NGINX. [5] This time, repeating step 2 should work. Signed-off-by: Galen Charlton <gmc@equinoxOLI.org> Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Created attachment 149641 [details] [review] Bug 32716: larger proxy_buffer_size in NGINX example config This patch updates the example NGINX config to increase the proxy_buffer_size to 16k. The default value of 4k (on some platforms) has empirically been shown to be a bit too small for the Link headers emitted by the REST API when pagination is requested. To test ------- [1] Set up a Koha system with NGINX as a reverse proxy in front of it (either in front of Apache or in front of of Starman). [2] Perform a patron search that returns at least two pages of results and navigate to the second page. [3] Note that the navigation can fail with a 502 HTTP error and an "upstream sent too big header while reading response header from upstream" error in the NGINX log. The problem is most likely when the pagesize of the server running NGINX is 4096 bytes. [4] Update the NGINX configuration per this patch and restart NGINX. [5] This time, repeating step 2 should work. Signed-off-by: Galen Charlton <gmc@equinoxOLI.org> Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Pushed to master for 23.05. Nice work everyone, thanks!
Nice work everyone! Pushed to stable for 22.11.x
Backported to 22.05.x for upcoming 22.05.13.
applied to 21.11 for 21.11.21