2026-08-14 19:20:32
tldr: the live version is here: https://curl.se/perf/
How fast is “fast” and is it good enough? Does it run as fast now as it did before or was there a regression? What exactly needs to be fast? How fast is it?
These are questions that many projects and products face, and in curl we are no different. Yet, performance testing and comparisons are hard and full of landmines and time-wasting efforts. For many years we have occasionally brought up the idea of a performance test suite for curl only to shut it down again because the challenges seemed hard and no one was volunteering to do this.
This week it changed.
I started out trying to find existing projects that host performance results for Open Source projects so that we could just feed our results something else and get great visualizations and data management. I did not find any such.
I then took a look at what existing tools there are for this purpose, and most pointers seemed to suggest that Grafana is a popular and maybe even a good solution to build something like this with. But man, that is a complicated machine and it felt more than a little overwhelming just figure out where or how to start with it. I decided to postpone that take as well.
I decided that instead of trying to do this the best and optimal way – I shouldn’t let perfect be the enemy of good – I would start out by doing the things I know how to do and take it as far as I can one step at a time. Something should be better than nothing.
Performance testing needs decently stable system conditions so that repeated runs produce reasonably similar results, when all involved factors remain identical. This is basically impossibly to accomplish using most cloud infrastructure since those are almost always shared with countless other users. At least on the cheap and free tiers we use.
We probably need our own dedicated hardware for this, but instead of trying to figure out where to get that and arrange for that, I would start by running performance tests on my own local development machine. I am a single user on this and it has many cores and runs decently fast. It should be good enough to get this going on.
I created a first shell script that updates the curl source code from git, it configures and builds it. Then it runs a bunch of tests, outputs a bunch of data and logs all the output in a single log file. I started out with a few simple tests. How fast does curl download a 100 GB file from localhost, how many allocations and how big allocations does it need for a single HTTP download?
My second script parses all the test log files from the previous builds and generates summaries and graphs for them. To make it possible for humans to see how the performance changes between builds and ideally to automatically detect when something changes more than what should be tolerated.
As I am a graph addict already since before, and that journey has taught me a little gnuplot, I decided that even while there probably are much better tools and fancy JavaScript things that could be used, I don’t know them and learning them now is an endeavor I rather avoid. So I stick to what I know and can get results with quickly.
A third script is invoked from a crontab every twenty minutes, sets up some variables and invokes the runner script.
Once the basics started to work, I showed my curl friends the early versions and I soon created a new git repository for the code.
After a little more poking, I soon made my locally produced performance test summary get packaged and automatically transferred to the curl website after each build, and voila, the first public curl performance tests were live and public.
Getting this data available immediate triggered curl developers. It only took hours until we had the first proposed changes to improve some numbers, and soon we had a few merges to that affect. Visibility really helps!
The performance numbers we get are still varying to a certain degree, partially of course because I still use my machine for my daily development things, but also because most of them do real (localhost) networking and that is by its nature a little… varying.
The system builds and runs a new round every twenty minutes and it does that using the latest commits from git. This setup makes it sometimes run many rounds on the same commit and it might also mean that it sometimes updates and get several new commits at once, so it might skip a round for some commits. I might reconsider this design later, but since it is still a twenty minute time window, the number of commits is still limited.
When the script makes multiple build rounds on the same commit, it accumulates the numbers and for the graph it stores the maximum, the median and the minimum value. It helps show the variation per commit and allows us to cram more into the graphs. It is still early days, but there will be a maximum limit to how many commits that can be displayed in a single graph and still be helpful.
To help visualize the distribution and data spread per test, I created a separate illustration that shows the minimum, maximum, P25, P75, medium and mean values in a Box-and-Whisker Plot.
An obvious downside with me just storing build logs in files, is that it will not scale up to the millions. I did however decide that I’m not designing this system for that. At least not now.
Performance tests are highly specific and dependent on the exact machine it runs on, the exact third party libraries and their versions that are used, the other components involved in the tests, such as the servers, and more.
I expect that we will change conditions for the tests every once in a while that makes it hard to compare the current numbers with past numbers. Therefore I think the performance test numbers and values are primarily useful in the short term. To help us spot if we land something that subtly and unintentionally degrades something.
To detect extremely slow and long-term changes in performance and even making sure we can better survive wiping all the existing build logs etc, I introduced a concept I call stakes. As in a stake pole. A marker. An arbitrary threshold set manually for each specific test. This value can be used to measure performance test results against, now and later. As conditions change and maybe something makes the results go up or down and we are fine with those changes because they are motivated and expected, then we just change the stakes.
If it works out, I might try to have the system automatically detect and maybe highlight tests that deviate too much from its set stake (at least if done in the wrong direction) . It could be a signal that something bad was merged.
As with everything in life, things are often balanced out. We already ran into this when we eagerly merged several changes to reduce the number of allocations done for a single HTTP download, only to realize that one of the optimizations we did had the side-effect that it expanded the size one of the main structs maybe a little too much…
Improvements in one area might come at an expense in another. With sufficient tests and data we can improve curl for users, and at the same time make sure that our changes don’t come with a cost we are not prepared to pay. Exactly how to make the balance is of course a question we need to deal with, discuss and decide. Possibly for every change we do!
As I write this, we have 24 tests and a full test round completes in about six minutes on my machine.
We can of course do multiple builds using different hardware, different operating systems, different build options, different third party libraries and different test servers to check more angles of performance, and I am certainly open for and prepared to do that going forward. I will however first let this single-flavor run for a while so that we get more data, get a change to tweak it and make it as usable as possible for curl developers.
As with everything there is no end to what we can make this do. This is a start. I sure we can take it further as we move along. In particular if people join in and help out. Both with ideas and proposals for visualizations, graphs and new tests to add, but also with actual pull-requests and code.
Over the last year, we have merged, on average, about 10 commits per day. If we keep this pace up and this performance test setup can show 100 build rounds conveniently into a single graph, that is just ten days of development. Probably not enough.
Once we reach one hundred builds or so in the first graphs I need to consider adding separate long term graphs that use select data-points to display data development over a longer time. Some googling told me the Largest-Triangle-Three-Buckets, or LTTB for short, is a fine algorithm to use for this. I now do a separate “long term” graph that “downsamples” the full range down to something that can be shown in a reasonable way. I suppose we will see properly in the future how this works.
The stake thing I mentioned is one way to help us spot gradual performance changes over time. Another googling told me that there’s a Mann-Kendall Test + Sen’s Slope algorithm to use to identify trends in graphs like this and it can be used to plot a trend. It might work as a helper to better identify… yeah, the data trend for each test.
This setup has only existed for a few days. There is lots to do, lots to learn and much more to experiment with.
Your comments, help and pull-requests will be appreciated!
2026-08-03 14:42:19
At this exact moment curl’s summer of bliss 2026 ends.
We (the maintainers of curl) took the entire month of July off from vulnerability reporting and in this post I will try to explain how this went.
(If you feel like skipping the wordy blab below, the single word answer is: fine)
This was possibly our best project decision in a long while.
Already before this, we have been refusing to answer emails about vulnerabilities. Partly because we can’t keep track of them that way but even more so because it makes it much harder to properly disclose and publish the entire report sequence after the fact.
On our Hackerone page we informed visitors that we were on pause and that they could come back in August.
We had I believe one vulnerability report sent to my private email address in this period in spite of that messaging, but for all intents and purposes this worked out exactly as good as we hoped it would. I just ignored that email. That was easy.
The effect was almost immediate. Just a few days into the bliss, my fellow curl maintainers all agreed with me that we felt a sense of relief, of vacation and that a load had been taken off our chests. We felt free, unchained, and now suddenly able to do what we wanted.
We could now spend time reviewing some of the queued up pull-requests for features and changes we like. We could suddenly again work on code in areas we had been leaving behind lately as vulnerability reports sucked all the air out the room. We polished details on the website, we found document gaps to tighten. It felt like the good old days again. The fun days. We got reminded why we do Open Source and how fun it is.
We took time off, saw some other corners of the world and enjoyed some time away from the keyboards.
We truly healed and re-energized.
Before we took off on the bliss, we were informed in clear terms that the CNA rules (we are a CNA) mandate that we must respond within 72 hours for some critical vulnerabilities so we can’t just ignore them. I told them sure we can, but in the worst case case our “root” could do some emergency assignments. I figured the risk was minimal and it turns out I was right, Nothing like that was needed and no CVE assignments were necessary during the bliss.
I got a curious question or two from existing support customers on how the bliss would affect them, but that was easy: it did not affect them. Now, post-bliss, I think they all can confirm that it really did not.
As I promised to keep up the contact with and support for paying customers even during the bliss, you could possibly imagine that this would have been an incentive for worried commercial curl users out there to sign up for support contracts.
This did not happen – at all. By this I think we should conclude that (commercial) curl users were not worried either.
Lots of fellow open source maintainers and most people in my surrounding have been super positive and downright supportive of our taking some time off. I can’t recall having receiving a single negative comment about the curl summer of bliss!
I was moved to see that several other Open Source projects followed our example and also took some time off in order to recharge and relax. In addition to giving us a little vacation, it helps sending a signal and a reminder that Open Source is to a large extent done voluntarily and even maintainers need a break at times.
Have we opened ourselves up for dangerous attacks and flaws now? Have the bad guys an edge on all curl users out there now because we lived in bliss for a month? We don’t know yet, but it would surprise me.
During this slow-down, we slowly got more open issues and pull-requests lingering on GitHub than usual. No surprise there. Once we started to come back to life again, we have since managed to return them back to the normal amounts.
Yes, there is an obvious risk that there are now a whole range of queued up reports that will hit us in a short period time as we open up for vulnerability reports again. Presumably the risk for duplicates among these reports should also be significantly higher than usual. I suppose I need to do an update post in a month or two and let you know what happened.
We always treat vulnerability reports and project security with topmost priority and we will continue to do so. We will simply work with what we have and make sure our users and by extension, the world, are safe.
Since I am a member of a few other (non-curl) security teams that did not have a summer of bliss, I have seen that the flood of vuln reports have not really slowed down so it might depend a lot on the details of each specific project.
All individual curl maintainers of course handled this gift in their own ways. We did not all just disconnect to sit on a remote beach for the whole time. Some of us did that part of the time, but we mostly enjoyed the lower stress level and the absence of pressure. It was mentally relaxing. So, even if some of us kept up with emails, occasionally responded to issues or even submitted some pull requests of our own, it was still vacation. It was still blissful.
Will we do another summer/winter of bliss? I think yes. It was simply great, with virtually no downsides for the people involved but instead lots of positiveness. Ideally a reduced workload going further will remove the need for another one, but it is not easy to tell what the future holds.
After all, curl just does transfers. Fast. Reliably. Secure.
2026-07-27 14:55:03
The recently published RFC 9421 describes how to do HTTP Message Signatures, and starting just now, curl experimentally supports them.
The specification describes this as a mechanism for creating, encoding, and verifying digital signatures or message authentication codes over components of an HTTP message. It is a way to verify that selected parts of the HTTP request arrives unmodified and exactly the same as when the request was created by the client.
These days, it is very common that there are layers of proxies, load balancers, front-ends, CDNs, web firewalls and what not in between the client and the ultimate application. With HTTP Message Signatures, there can be assurances that the headers are components of the request end are unaltered.
This functionality comes with four new command line options to allow users to use its full power:
--httpsig-algo allows the user to specify which algorithm to use, with ed25519 being used by default. The only other algorithm supported right now is hmac-sha256.
--httpsig-key specifies the key to use when signing the request.
--httpsig-keyid is the key identifier, a string that is passed on in the headers.
--httpsig-headers details exactly which parts of the request and which headers that should be signed. If not set, it defaults to signing the method, authority, path and query.
With these four new flags added to the list, curl supports 278 different command line options.
The corresponding options of course also exist as options for curl_easy_setopt:
CURLOPT_HTTPSIG_ALGORITHM: signing algorithm (“ed25519” or “hmac-sha256”)CURLOPT_HTTPSIG_KEY: the key to use for the signingCURLOPT_HTTPSIG_KEYID: key identifier for Signature-InputCURLOPT_HTTPSIG_HEADERS: a space-separated list of components to signThis feature is marked experimental. This means that it need to be explicitly enabled in the build to appear, and that we strongly discourage use of it in production as we reserve the rights to change it before it gets supported for real. We use the experimental phases as a time for people to test it, to tweak it and to learn what we should fix so that we then can support this to the end of time. We do not guarantee any backward compatibility for experimental features.
Please test this feature and tell us how you experienced it! The more tests and more feedback we get, the faster we can get moved out of the experimental phase to have it present for real for everyone.
This feature is already merged into git and will be part of the pending curl 8.22.0 release. As experimentally supported.
This feature was graciously brought to us by Sameeh Jubran.
Top image by Antonios Ntoumas from Pixabay
2026-07-25 22:29:28
It takes a village to make curl. A rather big village.
I have not been a solo maintainer of curl for a long time and I don’t even do half of the commits anymore
Since today, the curl git repository holds the accumulated efforts from 1,500 separate and named individuals. Only 4.5 years since we passed 1,000. Yay for us!
Author 1,500 turned out to be Sameeh Jubran who authored this.
2026-07-17 04:01:14
There is only one thing that is better than two days of HTTP workshop, and that is of course three days of HTTP workshop. The final day of this edition of the series started out with us again shuffling around where we parked ourselves around the big table. Except Mr captain of course who once again got to herd us forward through another day from the same seat.
MOQ (Media over QUIC transport) is not HTTP, but it uses QUIC so it is at least tangentially interesting and it involves a lot of the same people so this status update still felt welcome and suitable. Compared to existing HTTP based solutions, MOQ is supposed to offer less complexity and lower latency. The moon landing was broadcasted with less latency than current live-streamed TV and maybe MOQ can make us come close to those numbers again. In MOQ clients subscribe to a track that then contains a lot of objects that are delivered. It’s not the request + response approach of HTTP. The fact that this is not HTTP of course brings a lot of questions and well, doubts, and we lingered on various aspects of this topic for quite a while.
My prize for the best slides of the HTTP workshop 2026 goes to [redacted] for the excellent use of potato images in their presentation.
PTTH is HTTP spelled backwards, commonly pronounced as PoTaToH. A client sets up the connection but the actual HTTP request is sent from the server to the client. One of the intended use cases for this, is to allow an origin server to connect to the CDN proxy and then be able to deliver traffic to the world, rather than to have the CDN connect to the origin the way they usually do. Apparently most CDNs already have custom and proprietary solutions for exactly this kind of feature, so maybe doing it in a standard way instead makes sense?
The draft explains the new proposed way to continue a previously interrupted upload over HTTP. The upload request gets a Location: header back for the resource being uploaded, and if it gets stopped prematurely, a client can then HEAD that resource, figure out the size and then do a second upload (using the PATCH method) request that tells the server that this transfer should start at offset X.
Exactly how this should be supported in browser’ upload forms seemed a little bit uncertain. For my own sake I can see a challenge to implement this nicely for curl in particular when the upload is using formpost upload (curl’s -F flag) which after all still is a very common way to do uploads on the current web. I’ll return to this topic at a later time when I written an implementation to test…
io_uring is a Linux asynchronous I/O framework that avoids the overhead of traditional system calls. It uses two shared ring buffers between user space and the kernel, allowing applications to batch I/O operations with zero-copy efficiency.
The feature is disabled by Google in ChromeOS, Android and in production Google servers which certainly holds back some use of it.
io_uring can be helpful to speed up things, but might be complicated to use in existing software architectures and the presentation went into some details on why this is so.
A walk-through of some of the recent developments and improvements in Firefox’s UDP networking stack. Going from single datagrams to the modern ways to ship large chunks of data offloaded to the kernel to speed things up. Upload throughput in Firefox is up 60-90% over the last 11 releases. Lots of fun graphs and metrics were shown. This work is based on the quinn-udp stack.
Happy Eyeballs v3 is coming and Firefox is implementing it. It now takes into account many more data sources than before, including alt-svc and HTTPS-RR and races connections against each other to use the one that connects first. There are some recommended timers in the specification and parts of the discussion was around how maybe the timers could instead be tightened a bit, and maybe the delay between the subsequent attempts could then use an exponential backoff instead sticking to a fixed interval?
(I know I’ll discuss some of these details with my curl hacker friends and see what we should adjust… curl already supports most of the Happy Eyeballs v3 specification.)
As we approached the end of the day a few shorter topics were ventilated to give us a little more to consider before going home:
With this, the seventh HTTP workshop had ended. Again a very fine event. This time graciously sponsored and arranged by Adobe. Thank you everyone!
The general idea is to continue with these events roughly every second year and I support this. The HTTP workshops are definitely one of my favorite events.
The top image on this post was used in the final presentation and the author told me he is aware of the AI errors in there, “of which there are at least two”.
2026-07-16 04:07:14
If you missed it. I already described day one.
Caffeinated and ready, we all gathered in the same spacious room as yesterday, but seated in new places as “suggested” by our captain. Some of us even remembered to move over the name tags we wrote yesterday to our new seats.
No time was wasted on introductions today. We dove straight in at the deep end.
Is the future of software that we check-in the AI prompts in the git repository and trust it to generate the correct code? Are specifications the new level o
f abstraction for source code? These questions triggered long discussions with a huge mix of opinions and experiences getting shared about how AI is used, should be used and could be used now and in the future.
The Common Crawl spidering upgraded to using HTTP/2 for their scan and as an end result, I believe 61% of the responses used HTTP/2 and the entire round ended a few percent faster than before, which when you traverse a few billion URLs really makes a difference. They apparently use a locally patched version of Apache Nutch for this.
The HTTP probe project runs a lot of tests on HTTP/1 servers and compares how they behave in a lot of different aspects and then generates these awesome tables. Looks like something for every server implementer team to have a look at and decide what of these red boxes that should rather be converted into green alternatives.
HTTP Zoll is a new test suite for intermediaries that tests intermediaries (what we often call proxies) for a large amount of request and response smuggling issues. Some real world problems found were discussed and as this project aims at going Open Source words were expressed on what kind of precautions and checks that maybe should be done first. I hope we get to hear more about this project soon.
The HTTP Arena is another project that does performance and measurements. They test HTTP server frameworks and present the results in various ways on their site.
In this presentation, we were presented with different HTTP/3 deployment numbers from different sources and the associated reasoning around why they differ but then more importantly. what can and should be done to increase HTTP/3 usage.
Anti-virus interceptions, enterprise blocks and server-side performance not yet on par with TCP were mentioned as reasons for holding back the numbers.
Reasons for using HTTP/3 include use cases that encourage QUIC adoption: WebTransport, Media over QUIC and MASQUE (HTTP/3 proxies and HTTP/3 proxies over older HTTP proxies).
Using HTTPS-RR for upgrade was promoted, as every alt-svc response that is returned with an ALPN using h3 should perhaps also offer h3 over DNS. Why doesn’t your server announce its h3 support over HTTPS-RR?
QUIC v2 is deployed on an amazing 0.003% of all QUIC v1 domains and there was a discussion why this is so and the common sentiment in the room seemed to be that very few saw a reason for deploying v2 and several expressed a concern that doing so might in fact introduce issues. Someone (you can probably guess who) in the room increased that number a lot by quietly mentioning that haxproxy.org certainly supports it.
QUIC multiplexing over bi-directional streams is a proposal on how to do QUIC-style multiplexing over TLS (or anything else really). It has been adopted by the IETF QUIC working group and there was a somewhat extended discussion about what the HTTPbis group should or should not do with it. The biggest interest might be for data center use, but is that then something IETF should bother about? This is not the first time I blog about this, and even if there did not seem to be a strong demand or need for this, it also did not seem to be completely dead. I bet we will hear more about this later.
Doing a TLS terminating MITM proxy has its challenges and we were given some insights and experiences on the challenges of doing HTTP/2 and HTTP/3 to the server.
The browsers refuse to do HTTP/3 when they detect custom CA certs installed, which apparently is mostly because of lots of past bad experiences with anti-virus software that in particular seems to break QUIC and for users it is not obvious where the blame should go. This then makes browsers not do HTTP/3 over any MITM proxy.
Some time was spent on how allowing different clients to the proxy uses a shared h2 connection to the target server is complicated and not used, even though in theory it should be possible. An argument was made that it could even lead to worse performance than when using HTTP/1 but I could not quite follow that reasoning. I’m sure I missed some subtle detail in that explanation.
When the afternoon is running late and we have been promised beer and snacks after the final talk, what is better than a hard core technical presentation with lots of graphs and numbers showing how QUIC performance can be improved by tweaking the congestion control algorithm and send more data in the startup phase of a new QUIC connections? This new approach is called Rapid Start and it looks like a promising and yet simple improvement. According to experiments done on real world traffic, the time to last byte was reduced by 14.7% on average. Not bad at all.
Our meeting sponsor Adobe graciously sponsored drinks and food so we got to linger around for a few extra hours and talk even more HTTP and networking until the personal firmly insistent they needed us to leave the room and we instead continued solving world problems elsewhere. Topics around the table included the famous HTTP/2 spec coin flip, the QUIC spin bit, the SCONE situation for QUIC, the timeline behind the QUERY method and many more great stories.
Thanks for the beer!
Now we can’t wait for day three.
