If you can hard wire an ESP32 via Ethernet that would probably be best. You could also get better latency with an ESP32 over WiFi. However, a ZigBee button should be pretty low latency already.
Anecdotally I get perceptibly instant response from an ESP32 over WiFi.
It's probably marginally faster from the dashboard. This isn't only about the ZigBee delay though - it's really perfectly OK. Reducing latency is as much for the fun of it than anything else. I'm interested in knowing what the fastest possible input method is.
Theoretically, MQTT would be faster than HTTP; with MQTT the clients all maintain a persistent connection to the server, so sending a message is just a case of sending another packet on the existing connection, where as HTTP in the worst case would require a DNS lookup, TCP connection setup and TLS handshake before the button press could be reported.
The downside of a persistent connection is you have to maintain a persistent connection - not an issue if you are hardwired, but for things in batteries it probably isn't practical
I have a wired device sending HTTP POST updates very regularly (often more than one per second) and if I watch those arrive, they appear almost instantaneous. If the sending device used IP (or, more likely, had cached the lookup) I guess that would be fast too.
Practically it doesn't make any real difference - unless your network is ancient/massively overloaded/completely screwed or your client device is low power enough to struggle with TLS the whole connection setup process is only going to take ~100ms at most
OP asks for "lowest latency" but HTTP is fast enough, and later in the thread says "perceptibly instant" is the goal. HTTP is going to be your slowest option, almost certainly, but even the slowest solution is "instant".
What exactly are you trying to do, OP?
(Esphome seems like the answer you're looking for. It's faster than MQTT and you can program parts of the automation and service management in the ESP hardware. RF is probably the only way to go faster.)
Zigbee is for this usecase instant and definitely faster than any ESP wifi based solution. Your slowness here isn't for reaching HA, it's HA itself and primarily pushover and Alexa. What is it you are trying to do? For anything that should feel instant, getting rid of pushover and Alexa will be crucial.
I don't think that MQTT could add any perceptible latency. Maybe is the button itself that adds it.
Isn't the button able to recognize multiple pushes? If yes, there lies the delay: the button waits for a second push and if it doesn't arrives, it sends out the command to the coordinator.
What kind of ZigBee button are you using?
That's a good point; the button does support double push, I might be able to disable that. It's some old unit I picked up for next to nothing, I have some Aqura buttons about to try.