Are there any good self hosted software that can create subtitles for videos?
All the solutions I'm seeing are some third party service where I would have to upload my videos to them to get them transcribed.
Here's one I've been playing with https://github.com/jhj0517/Whisper-WebUI
The small model of fast Whisper has been amazing for the 3 options it gives (files, YT, or recording), tho I have in mind the limitations and I've only used it with somewhat clear audio.
I second this. Also, I have been experiencing with automating it, using https://hub.docker.com/r/onerahmet/openai-whisper-asr-webservice as a provider for Bazarr
That looks pretty much what I'm looking for, thank you! I see instructions to run in Windows, but does it also work on Macs?
If you need it for media that has subtitles available somewhere, then there are plugins for that (ex. Jellyfin/Plex/Kodi)
If you're looking for something to automatically transcribe audio locally, I'm not as sure but others already suggested some
Which are you looking for?
Maybe you don't even need that, at least for accessibility.
Windows for example now has exactly this feature, which is a speech-to-text-transformer powered by some "AI".
But, in contrast to the Bing chat, this works (afaik) offline by some FOSS-backend, which I don't know the name of anymore (maybe someone else will?) You could use that tool for live transcription. That is supposed to work extremely well!
Please correct my if I'm wrong, I don't use Windows anymore personally, and at work we have a business edition that doesn't ship this brand new feature yet.
(Side note: as strongly as I hate Windows, this feature is absolutely godsend for hearing-impaired people and should be adopted by every other OS!)
If you want to transcript movies and thereof in bulk by uploading them, I can't give you any information, sorry.
But I believe there are some sites that give you the "subtitle file" for download freely, which you can add manually for each movie in Plex/ Jellyfin.
Android does on-device transcription of any Audio source as well in recent versions!
The issue with providing this with open-source software is that it tends to require deep integration into the OS, which needs pretty much the same kinds of APIs that spyware also needs, so they get locked down a lot ...
For example on Android I'm pretty sure that a 3rd party play store app could not provide the same feature without requiring the user to click through some unavoidable, scary sounding warnings from the OS (if at all).
AFAIK iOS and macOS have had this for some time now. I think since a couple years back it’s all local-only on newer devices as well.
Might be janky, but if you really wanted this for free you could get a speech to text program like futo, play the video and have it transcribe it and save it to a text file, then copy and paste in the subtitles
Maybe Whishper would be suitable?
Okay yeah, I spun up a docker instance and this is cool as fuck. It seems to be exactly what OP is looking for. This is cool enough to be a post on its own tbh. It would be perfect in a ytdl workflow, as you can do the transcription by linking a video. I’ve been holding off on adding youtube to my Jellyfin setup for just this sort of tool. I hope the add the GPU accelerated faster-whisper models soon.
Luckily I still had the project in my history! Glad it was useful.
That looks perfect! Thank you!
Oh that looks really cool, thank you for the link.