ETag and caching for serving pre‐compressed files #216
Conversation
This PR introduces ETag handling and caching improvements for serving pre‐compressed .gz files, enabling conditional GET responses to optimize bandwidth and server resource usage. Adds ETag header generation based on the CRC32 checksum from the gzip trailer. Implements conditional 304 responses in the send() method when the client's ETag matches the server's. Enhances both asynchronous file response handling and web server request processing for .gz files.
src/ESPAsyncWebServer.h
Outdated
| @@ -334,6 +334,8 @@ class AsyncWebServerRequest { | |||
| void addInterestingHeader(__unused const String &name) { | |||
| } | |||
|
|
|||
| static void _getEtag(uint8_t trailer[4], char* serverETag); | |||
There was a problem hiding this comment.
Has to be moved to private: section
There was a problem hiding this comment.
_getEtag is declared and used in AsyncWebServerRequest. But it's also used in AsyncFileResponse.
If I set it to private, it wouldn't be accessible in the AsyncFileResponse, since it's not a "friend" (or at least I don't know how to do it).
Could you help me?
There was a problem hiding this comment.
There are examples if I remember.
But it has to be moved private because if put i na public API, it is something that has to be part of the API. We had some issues already with some methods that are public when they should not be.
There was a problem hiding this comment.
@JosePineiro : try and let me know if you have issues doing that. I can hopefully append a commit n your branch.
There was a problem hiding this comment.
The method _getEtag is in one class and must be called from another, which isn't possible if it's private. The method name begins with "_", which indicates that it should not be used outside the library.
I can think this possibilities:
1 - Create a copy of the method in the other class, but you've already rejected this option (breaking the DRY principle).
2 - Make the class a "friend," but that breaks encapsulation.
3 - Give me an example of how to do it, because I can't think of any other option, Function pointers to bypass encapsulation are a very bad idea.
Tell me which option seems acceptable to you. If neither of them is valid, please, closes the "Pull requests," because I won't be able to meet this requirement. I'm sorry for wasting your time.
There was a problem hiding this comment.
2 is ok :-) I agree this is not clean but that's what's used so far in the lib.
Another option is to create a utility class in a separate file (public of course) documented as such that users do not use it.
As long as it is really clear to any user to not use at all this function...
thank :-)
There was a problem hiding this comment.
I think that making a method or variable start with "_" is a good indication that it shouldn't be used outside of the library.
It's probably best to restructure the library classes so that "friend" isn't needed, but I understand that maintaining backward compatibility is very important.
I'll provide you with option 2. It's KISS compliance and doesn't compromise future refactorings.
Another question: Is it better to create an issue first for future changes?
There was a problem hiding this comment.
thanks for the update!
no need for issues: issues are for bugs. To discuss an implementation, the discussion forum is available ;-)
|
Very cool! In WLED we've got similar logic being applied to statically linked resources, ie. pre-gzipped content stored in static variables and served via |
When this commit is finished merging, create another branch to implement ETag handling in the void send(int code, const char *contentType, const uint8_t *content, size_t len, AwsTemplateProcessor callback = nullptr) function |
Make private: static void _getEtag(uint8_t trailer[4], char *serverETag)
|
Thank you @JosePineiro for making this PR ready to go :-) |
This PR brings improved standards-compliant handling of pre-compressed .gz files, providing ETag support and conditional responses, helping optimize client-side caching and server resource usage. Fully backward compatible.
ETag Generation for Pre-compressed .gz Files
When a .gz file is served via AsyncFileResponse, the server now generates an ETag header based on the CRC32 checksum found ((first 4 bytes) in the gzip trailer (last 8 bytes of gz file).
A Cache-Control: no-cache header is also added to enforce proper cache validation.
Conditional Request Support in send()
The send() method now checks for the If-None-Match header when a .gz version of the requested file exists.
If the client's ETag matches the server's ETag (derived from the gzip trailer), the server responds with 304 Not Modified, saving bandwidth and improving efficiency.
If the ETag does not match or no valid ETag is present, the normal file response is sent.
These changes only apply to pre-compressed .gz files and do not alter the behavior for uncompressed files.
Why this is useful?
For static resources, improves client-side caching efficiency by allowing conditional requests with If-None-Match
Reduces unnecessary re-downloads of static resources when they haven't changed
Aligns with common web best practices for serving compressed assets with proper cache validation.
ETag values are derived from the CRC32 checksum embedded in the gzip trailer, ensuring the ETag reflects the original uncompressed content. If change the uncompressed content, change the Etag.
It is fully backward compatible. It has no adverse effects on library users and does not require them to modify their code.