
I love to put a progress bar on the website.
It’s very hard to track processes on the client-side. Since my backend is a single-thread process, this operation can block other requests for a while and may get timed out during this time. Generating a zip file takes a long time. With the approach above, I need to redo the whole thing all the time, it’s not really efficient. So, while the Zip file itself is not likely reusable, the single files do. Although the Zip files are different sets of reports, it’s very likely that most of the individual reports have been generated by an earlier request. It cannot utilize the feature I had already built. They will take time to process and require a lot of effort coding, testing, and maintenance later on. I need to consider generating all files per request or combining between reusing existing files and generating new ones. The logic to generate a zip file is quite complicated. Instead of individual files, I needed to download a zip file containing hundreds of reports. If a user orders the same thing, the backend will return the URL of the existing file.Ī few days ago, I had a new requirement.
Since generating a report takes a while, the output files are stored and the server caches their URLs by request params.
For each request, our backend will generate a report, upload it to Amazon S3 storage, and return its URL to the client. I recently worked on a side project, which generates reports per user’s request.