Compare commits

..

10 Commits

Author SHA1 Message Date
Marek Lenczewski
2156bb3226 update 2026-04-08 19:15:34 +02:00
Marek Lenczewski
a0c8ecaf27 update 2026-04-08 08:51:59 +02:00
Marek Lenczewski
375a9cd386 update 2026-04-07 18:01:34 +02:00
Marek Lenczewski
ca988345e9 update 2026-04-07 17:55:30 +02:00
Marek Lenczewski
8f15f51bce update 2026-04-07 16:13:16 +02:00
Marek Lenczewski
52c4e5f33d update 2026-04-07 15:50:50 +02:00
Marek Lenczewski
eb71c148d5 update 2026-04-06 22:07:11 +02:00
Marek Lenczewski
8e1d482ef2 update 2026-04-06 21:40:43 +02:00
Marek Lenczewski
f48f0aac08 update 2026-04-06 20:37:53 +02:00
Marek Lenczewski
2f37d1b31f update 2026-04-06 20:01:19 +02:00
53 changed files with 683 additions and 540 deletions

117
CLAUDE.md
View File

@@ -9,6 +9,7 @@ Selbst-gehostete Anwendung: YouTube-Videos per Browser Extension erfassen, auf e
- `features.md` — Konkrete Features und Benutzerinteraktionen
- `architecture.md` — Detaillierter Systemaufbau (Endpoints, Services, Models, Screens)
- `szenarios.md` — Benutzer-Szenarien
- `modules.md` — Modul-Aufteilung pro Komponente
## Architektur
@@ -16,31 +17,45 @@ Drei Komponenten:
### Browser Extension (`browser_extension/`)
- **Manifest V2**, Firefox-kompatibel (`browser.*` API)
- `content.js` — extrahiert Videodaten direkt aus dem YouTube-DOM:
- Modulare Ordnerstruktur (`tracking/`, `api/`, `config/`); `manifest.json` bleibt im Root
- `tracking/content.js` — extrahiert Videodaten direkt aus dem YouTube-DOM:
- `ytd-rich-item-renderer` (Homepage, Abos, Kanalseiten)
- `ytd-video-renderer` (Suchergebnisse)
- IntersectionObserver (threshold 50%) — nur sichtbare Cards erfassen
- MutationObserver registriert neue Cards beim IntersectionObserver
- `yt-navigate-finish` Event-Listener fuer SPA-Navigation
- Deduplizierung ueber `sentUrls` Set, wird bei Navigation geleert
- Batch-Versand: sammelt sichtbare Videos mit Profil-ID, sendet als Array
- **Einzelversand**: jedes sichtbare Video sendet sofort einen separaten POST (kein Batching mehr)
- Selektoren ohne Klassen: nur Tags (`h3`, `img`) und Attribute (`href`, `src`)
- `background.js` — empfaengt Batch vom Content Script, sendet POST an Server
- `popup.html/popup.js` — Profil-Auswahl (holt Profile vom Server, speichert in browser.storage.local)
- `api/background.js` — empfaengt `{profileId, video}` vom Content Script, baut URL `/profiles/{profileId}/videos` und sendet POST
- `config/popup.html` + `config/popup.js` — Profil-Auswahl (holt Profile vom Server, speichert in browser.storage.local)
- Laden via `about:debugging#/runtime/this-firefox` → "Temporaeres Add-on laden" → `manifest.json`
### Server (`backend/`)
- **Python, FastAPI, SQLAlchemy, SQLite** (`videos/youtubeapp.db`)
- **yt-dlp + ffmpeg** fuer Video-Download und Streaming
- **WebSocket** (`/ws`) — benachrichtigt verbundene Clients bei neuen Videos
- **yt-dlp + ffmpeg + Deno** fuer Video-Download und Streaming
- **WebSocket** (`/ws`) — benachrichtigt verbundene Clients bei neuen Videos (sendet einzelne `profileId`)
- Dockerisiert: `docker compose up --build -d` im `backend/` Verzeichnis
- Laeuft auf `http://localhost:8000`
- Download-Service speichert Videos unter `/videos/{id}.mp4`
- Stream-Service: heruntergeladene Videos von Datei, sonst progressiver Download via yt-dlp mit gleichzeitigem Streaming
- Dedup: beim Batch-Import wird bestehender Eintrag mit gleicher Video-ID geloescht und neu eingefuegt
- Sortierung: nach ID absteigend (erstes Video im Batch bekommt hoechste ID)
- Profile: fest in DB definiert, Videos ueber Many-to-Many zugeordnet
- Nach lokalem Download wird die Server-Datei geloescht (file_path auf null)
- Modulare Ordnerstruktur (`database/`, `model/`, `api/`, `download/`, `stream/`, `notify/`, `base/`) mit `__init__.py`-Markern
- Entrypoint: `base.app:app` (im Dockerfile als `uvicorn`-Target)
- **Active Record Pattern**: DB-Methoden leben als Klassenmethoden auf den Models, nicht in einem Service-Layer
- Download-Service: `downloadVideo` ruft yt-dlp mit `--force-overwrites --no-continue`, prueft Datei-Existenz und Mindestgroesse, dann `Video.updateFilePath`. `downloadAsync` startet das im Hintergrund-Thread
- Stream-Service: `streamAndSave` ist ein Generator, der `streamVideoLive` (ffmpeg Live-Muxing) umhuellt und am Ende `Video.updateFilePath` setzt
- **Profil-scoped Dedup**: pro Profil eine eigene Video-Zeile. `Video.deleteIfExists(db, youtubeUrl, profileId)` loescht nur die Zeile dieses Profils, `Video.create(...)` fuegt eine neue ein. Profile sind unabhaengig.
- Sortierung: nach ID absteigend
- Profile: fest in DB definiert, Videos ueber Many-to-Many zugeordnet (de facto 1:1 pro Zeile durch das per-Profil-Dedup)
- Nach lokalem Download wird die Server-Datei geloescht (`Video.deleteServerFile`)
#### Naming-Konventionen im Backend
- **camelCase** fuer Variablen, Funktionen, Methoden, Pydantic-Felder, Klassenattribute (auch in `Video`-SQLAlchemy-Model — DB-Spaltennamen werden via `Column("snake_case", ...)` angegeben)
- **PascalCase** fuer Klassen
- **UPPER_SNAKE_CASE** fuer Konstanten (`VIDEOS_DIR`, `CHUNK_SIZE`, `DATABASE_URL`)
- **snake_case** bleibt nur fuer DB-Tabellen-/Spaltennamen im SQL und Python-Standardbibliothek
- Verstoesst gegen PEP 8, ist vom User explizit so gewuenscht (Hintergrund Symfony/Spring Boot)
#### DB-Session per FastAPI
- `DbSession = Annotated[Session, Depends(getDb)]` in `database/database.py` definiert; Routen schreiben nur `db: DbSession`
### App (`app/`)
- **Kotlin, Jetpack Compose**, Android/Android TV
@@ -50,48 +65,62 @@ Drei Komponenten:
- OkHttp WebSocket-Client — automatisches Neuladen bei neuen Videos
- Navigation mit TopBar (Profil-Auswahl, Aufraeumen-Icon) und Bottom Bar, Dark Theme
- Profil-Auswahl wird in SharedPreferences persistiert, filtert Videos nach Profil
- Lokaler Download: Videos werden auf dem Geraet gespeichert, lokal bevorzugt abgespielt
- Lokaler Download: Videos und Metadaten werden auf dem Geraet gespeichert, lokal bevorzugt abgespielt, offline verfuegbar
- Heruntergeladen-Tab liest aus lokalem Storage, **nicht** vom Server (kein `/downloaded`-Endpoint mehr)
- Aufraeumen: loescht alle nicht lokal gespeicherten Videos des Profils (sendet lokale IDs als Ausnahme)
- Server-IP konfigurierbar in `ApiClient.kt` (aktuell `192.168.178.92`)
- Emulator: Android Studio → Device Manager → Pixel 6a, API 35
- Server-IP konfigurierbar in `ApiClient.kt` (Emulator: `10.0.2.2`, echtes Geraet: `192.168.178.34`)
- Emulator: Android Studio → Device Manager → Pixel 7a, API 36
- **camelCase** Felder durchgaengig (`thumbnailUrl`, `youtubeUrl`, `isDownloaded`, `profileIds`)
## API Endpoints
Profil-bezogene Routen liegen in `api/profile_controller.py`, video-Aktionen in `api/video_controller.py`.
- `GET /profiles` — alle Profile abrufen
- `POST /videos` — Video-Batch von Extension empfangen (Dedup, Reverse-Insert, Profil-Zuordnung, WebSocket-Benachrichtigung)
- `GET /videos` alle Videos abrufen (optional `?profile_id=X`, sortiert nach ID absteigend)
- `GET /videos/downloaded` — heruntergeladene Videos abrufen (optional `?profile_id=X`)
- `DELETE /videos?profile_id=X&exclude_ids=` — Videos des Profils loeschen (ausser lokal gespeicherte)
- `POST /videos/{id}/download` — Download auf Server triggern
- `GET /videos/{id}/stream` — Video streamen (von Datei oder progressiver Download via yt-dlp)
- `GET /videos/{id}/file`Video-Datei zum Download auf Client ausliefern
- `DELETE /videos/{id}/file` — Server-Datei loeschen (nach lokalem Download)
- `WS /ws` — WebSocket, sendet Profile-IDs bei neuen Videos
- `POST /profiles/{profileId}/videos` — Einzelnes Video von Extension empfangen (Dedup pro Profil, WebSocket-Benachrichtigung). Antwort: `204 No Content`
- `GET /profiles/{profileId}/videos` — Videos eines Profils, sortiert nach ID absteigend
- `POST /profiles/{profileId}/videos/cleanup` — Videos des Profils loeschen (Body: `{"excludeIds": [...]}` fuer lokal gespeicherte Ausnahmen)
- `POST /videos/{id}/download` — Download auf Server triggern (Background-Thread via `downloadAsync`)
- `GET /videos/{id}/stream` — Video streamen (von Datei oder Live-Muxing via `streamAndSave`)
- `GET /videos/{id}/file` — Video-Datei zum Download auf Client ausliefern (`Video.getValidPath` korrigiert verwaiste DB-Eintraege)
- `DELETE /videos/{id}/file`Server-Datei loeschen (`Video.deleteServerFile`)
- `WS /ws` — WebSocket, sendet eine einzelne `profileId` als Text bei neuen Videos
JSON-Wire-Format ist durchgaengig **camelCase** (Backend, Extension, App).
## Projektstruktur
```
backend/
main.py — FastAPI App, CORS, Startup, Seed-Profile, WebSocket
database.py — SQLAlchemy Engine, Session, Base
models.py — Video, Profile, video_profiles (Many-to-Many)
schemas.py — Pydantic Schemas (VideoCreate, VideoResponse, ProfileResponse)
routes/videos.py — Video- und Profil-Routen
services/
video_service.py — CRUD-Operationen, Dedup, Profil-Filter
download_service.py — yt-dlp Download
stream_service.py — Progressiver Download + Streaming via yt-dlp
Dockerfile — Python 3.12 + ffmpeg
base/
app.py — FastAPI App, CORS, Startup, Seed-Profile, Router-Includes
database/
database.py — SQLAlchemy Engine, SessionLocal, Base, getDb, DbSession-Alias
model/
profile.py — Profile-Klasse, getAll() liefert list[dict]
video.py — Video-Klasse mit Properties (isDownloaded, profileIds) und Klassenmethoden (deleteIfExists, create, getAll, getById, updateFilePath, getValidPath, deleteServerFile, deleteNotDownloaded)
profile_video.py — videoProfiles M:N-Tabelle
api/
schemas.py — Pydantic Schemas (VideoCreate, VideoResponse, CleanupRequest)
profile_controller.py — /profiles und profil-scoped Video-Routen
video_controller.py — Video-Aktionen (download, stream, file, deleteFile)
download/
download_service.py — yt-dlp Download (downloadVideo, downloadAsync)
stream/
stream_service.py — ffmpeg Live-Muxing (streamVideoLive, streamAndSave)
notify/
notify_clients.py — WebSocket-Endpoint, connectedClients, notifyClients(profileId), registerWebsocket
Dockerfile — Python 3.12 + ffmpeg + Deno, CMD `uvicorn base.app:app`
docker-compose.yml — Service-Definition, Port 8000, Volume /videos
.dockerignore — videos/, __pycache__/
.gitignore — videos/, __pycache__/
browser_extension/
manifest.json — Manifest V2, Permissions, browser_action, storage
content.js — DOM-Extraktion + IntersectionObserver + Batch-Versand mit Profil
background.js — Batch-POST an Server
popup.html — Profil-Auswahl UI
popup.js — Profile laden, Auswahl speichern
manifest.json — Manifest V2, Permissions, browser_action, storage; verweist auf tracking/, api/, config/
tracking/content.js — DOM-Extraktion + IntersectionObserver + Einzel-Send mit Profil
api/background.js — Einzel-POST an /profiles/{profileId}/videos
config/popup.html — Profil-Auswahl UI
config/popup.js — Profile laden, Auswahl speichern
app/
.gitignore — .gradle/, build/, .idea/, local.properties
@@ -116,6 +145,14 @@ app/
- Server-Datei wird nach lokalem Download geloescht — spart Speicherplatz auf dem Server
- DOM-Extraktion statt ytInitialData-Parsing — funktioniert auch bei SPA-Navigation und Scrollen
- IntersectionObserver statt blindem Scan — nur sichtbare Videos erfassen
- Progressiver Download via yt-dlp mit gleichzeitigem Streaming statt komplettem Download vor dem Abspielen
- ffmpeg Live-Muxing statt komplettem Download vor dem Abspielen
- Deno als JavaScript-Runtime fuer yt-dlp — YouTube erfordert JS-Ausfuehrung zur URL-Extraktion
- Videos ohne Profilzuweisung werden automatisch dem Standardprofil zugeordnet (Fallback in `notifyClients` und `Video.create`)
- Per-Profil-Dedup statt globalem Dedup — verhindert, dass Profile sich gegenseitig Videos loeschen
- Einzelversand statt Batches — einfachere Logik, geringfuegig schlechtere Sortier-Garantie wird in Kauf genommen
- WebSocket statt Polling — effiziente Echtzeit-Aktualisierung der Videoliste
- REST Nested Resources — `profileId` in der URL statt als Query/Body
- Active Record statt Service-Layer — DB-Methoden direkt am Model
- camelCase statt snake_case im Python-Code — bewusster Verstoss gegen PEP 8 zugunsten der Lesbarkeit fuer den User (Hintergrund Symfony/Spring Boot)
- Sprache der Dokumentation: Deutsch
```

View File

@@ -16,6 +16,23 @@ android {
versionName = "1.0"
}
signingConfigs {
create("release") {
storeFile = file("../release-key.jks")
storePassword = "youtubeapp"
keyAlias = "youtubeapp"
keyPassword = "youtubeapp"
enableV1Signing = true
enableV2Signing = true
}
}
buildTypes {
getByName("release") {
signingConfig = signingConfigs.getByName("release")
}
}
buildFeatures {
compose = true
}

View File

@@ -15,7 +15,8 @@
<activity
android:name=".MainActivity"
android:exported="true">
android:exported="true"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />

View File

@@ -1,11 +1,14 @@
package com.youtubeapp.data
import android.os.Build
import retrofit2.Retrofit
import retrofit2.converter.gson.GsonConverterFactory
object ApiClient {
// Server-IP hier anpassen
const val BASE_URL = "http://marha.local:8000/"
val BASE_URL: String = if (Build.FINGERPRINT.contains("generic") || Build.FINGERPRINT.contains("sdk"))
"http://10.0.2.2:8000/"
else
"http://192.168.178.34:8000/"
val api: VideoApi by lazy {
Retrofit.Builder()

View File

@@ -1,12 +1,15 @@
package com.youtubeapp.data
import android.content.Context
import com.google.gson.Gson
import java.io.File
import java.io.FileOutputStream
import java.net.URL
class LocalStorageService(private val context: Context) {
private val gson = Gson()
private fun videosDir(): File {
val dir = File(context.filesDir, "videos")
if (!dir.exists()) dir.mkdirs()
@@ -15,6 +18,8 @@ class LocalStorageService(private val context: Context) {
private fun videoFile(videoId: Int): File = File(videosDir(), "$videoId.mp4")
private fun metadataFile(videoId: Int): File = File(videosDir(), "$videoId.json")
fun isLocallyAvailable(videoId: Int): Boolean = videoFile(videoId).exists()
fun getLocalFile(videoId: Int): File? {
@@ -22,7 +27,10 @@ class LocalStorageService(private val context: Context) {
return if (file.exists()) file else null
}
fun deleteLocalFile(videoId: Int): Boolean = videoFile(videoId).delete()
fun deleteLocalFile(videoId: Int): Boolean {
metadataFile(videoId).delete()
return videoFile(videoId).delete()
}
fun getLocalVideoIds(): List<Int> {
return videosDir().listFiles()
@@ -31,6 +39,24 @@ class LocalStorageService(private val context: Context) {
?: emptyList()
}
fun saveMetadata(video: Video) {
metadataFile(video.id).writeText(gson.toJson(video))
}
fun getLocalVideos(): List<Video> {
return videosDir().listFiles()
?.filter { it.extension == "json" }
?.mapNotNull { file ->
try {
val video = gson.fromJson(file.readText(), Video::class.java)
if (videoFile(video.id).exists()) video else null
} catch (_: Exception) {
null
}
}
?: emptyList()
}
fun downloadAndSave(videoId: Int): File {
val url = "${ApiClient.BASE_URL}videos/$videoId/file"
val file = videoFile(videoId)

View File

@@ -4,10 +4,10 @@ data class Video(
val id: Int,
val title: String,
val youtuber: String,
val thumbnail_url: String,
val youtube_url: String,
val is_downloaded: Boolean,
val profile_ids: List<Int> = emptyList()
val thumbnailUrl: String,
val youtubeUrl: String,
val isDownloaded: Boolean,
val profileIds: List<Int> = emptyList()
)
data class Profile(

View File

@@ -3,20 +3,19 @@ package com.youtubeapp.data
import retrofit2.http.GET
import retrofit2.http.POST
import retrofit2.http.Path
import retrofit2.http.Query
interface VideoApi {
@GET("videos")
suspend fun getAllVideos(@Query("profile_id") profileId: Int? = null): List<Video>
@GET("videos/downloaded")
suspend fun getDownloadedVideos(@Query("profile_id") profileId: Int? = null): List<Video>
@GET("profiles/{profileId}/videos")
suspend fun getAllVideos(@Path("profileId") profileId: Int): List<Video>
@POST("videos/{id}/download")
suspend fun triggerDownload(@Path("id") id: Int): Map<String, String>
@POST("videos/cleanup")
suspend fun cleanupVideos(@retrofit2.http.Body body: Map<String, @JvmSuppressWildcards Any>): Map<String, Int>
@POST("profiles/{profileId}/videos/cleanup")
suspend fun cleanupVideos(
@Path("profileId") profileId: Int,
@retrofit2.http.Body body: Map<String, @JvmSuppressWildcards Any>,
): Map<String, Int>
@retrofit2.http.DELETE("videos/{id}/file")
suspend fun deleteServerFile(@Path("id") id: Int): Map<String, String>

View File

@@ -2,12 +2,9 @@ package com.youtubeapp.data
class VideoRepository(private val api: VideoApi = ApiClient.api) {
suspend fun getAllVideos(profileId: Int? = null): Result<List<Video>> =
suspend fun getAllVideos(profileId: Int): Result<List<Video>> =
runCatching { api.getAllVideos(profileId) }
suspend fun getDownloadedVideos(profileId: Int? = null): Result<List<Video>> =
runCatching { api.getDownloadedVideos(profileId) }
suspend fun triggerDownload(videoId: Int): Result<String> = runCatching {
val response = api.triggerDownload(videoId)
response["status"] ?: "unknown"
@@ -19,8 +16,8 @@ class VideoRepository(private val api: VideoApi = ApiClient.api) {
}
suspend fun cleanupVideos(profileId: Int, excludeIds: List<Int>): Result<Int> = runCatching {
val body = mapOf("profile_id" to profileId, "exclude_ids" to excludeIds)
val response = api.cleanupVideos(body)
val body = mapOf("excludeIds" to excludeIds)
val response = api.cleanupVideos(profileId, body)
response["deleted"] ?: 0
}

View File

@@ -40,7 +40,7 @@ fun VideoCard(video: Video, onClick: () -> Unit) {
) {
Column {
AsyncImage(
model = video.thumbnail_url,
model = video.thumbnailUrl,
contentDescription = video.title,
contentScale = ContentScale.Crop,
modifier = Modifier

View File

@@ -83,7 +83,7 @@ fun VideoDetailScreen(
Column(modifier = Modifier.fillMaxSize()) {
AsyncImage(
model = video.thumbnail_url,
model = video.thumbnailUrl,
contentDescription = video.title,
contentScale = ContentScale.Crop,
modifier = Modifier
@@ -121,7 +121,7 @@ private fun VideoInfo(
Text(
text = when {
isLocal -> "Lokal gespeichert"
video.is_downloaded -> "Auf Server heruntergeladen"
video.isDownloaded -> "Auf Server heruntergeladen"
else -> "Noch nicht heruntergeladen"
},
style = MaterialTheme.typography.bodySmall,

View File

@@ -103,30 +103,22 @@ class VideoViewModel : ViewModel() {
}
fun loadAllVideos() {
val profileId = _state.value.selectedProfileId ?: return
viewModelScope.launch {
_state.value = _state.value.copy(isLoading = true, error = null)
repository.getAllVideos(profileId = _state.value.selectedProfileId)
repository.getAllVideos(profileId)
.onSuccess { videos ->
_state.value = _state.value.copy(allVideos = videos, isLoading = false)
}
.onFailure { e ->
_state.value = _state.value.copy(error = e.message, isLoading = false)
.onFailure {
_state.value = _state.value.copy(error = "Server nicht erreichbar", isLoading = false)
}
}
}
fun loadDownloadedVideos() {
viewModelScope.launch {
_state.value = _state.value.copy(isLoading = true, error = null)
repository.getAllVideos(profileId = _state.value.selectedProfileId)
.onSuccess { videos ->
val local = videos.filter { localStorage?.isLocallyAvailable(it.id) == true }
_state.value = _state.value.copy(downloadedVideos = local, isLoading = false)
}
.onFailure { e ->
_state.value = _state.value.copy(error = e.message, isLoading = false)
}
}
val videos = localStorage?.getLocalVideos() ?: emptyList()
_state.value = _state.value.copy(downloadedVideos = videos)
}
fun triggerDownload(videoId: Int) {
@@ -144,16 +136,25 @@ class VideoViewModel : ViewModel() {
val status = result.getOrNull()
if (status == "download_started") {
while (true) {
val profileId = _state.value.selectedProfileId ?: 1
var attempts = 0
while (attempts < 150) {
delay(2000)
val videosResult = repository.getAllVideos()
val videosResult = repository.getAllVideos(profileId)
val video = videosResult.getOrNull()?.find { it.id == videoId }
if (video?.is_downloaded == true) break
if (video?.isDownloaded == true) break
attempts++
}
if (attempts >= 150) throw Exception("Download fehlgeschlagen")
}
localStorage?.downloadAndSave(videoId)
val video = _state.value.allVideos.find { it.id == videoId }
if (video != null) {
localStorage?.saveMetadata(video)
}
repository.deleteServerFile(videoId)
loadDownloadedVideos()
_state.value = _state.value.copy(
isDownloading = false,
downloadStatus = "Lokal gespeichert"
@@ -187,6 +188,7 @@ class VideoViewModel : ViewModel() {
fun getVideoById(videoId: Int): Video? {
return _state.value.allVideos.find { it.id == videoId }
?: _state.value.downloadedVideos.find { it.id == videoId }
}
fun isLocallyAvailable(videoId: Int): Boolean {

BIN
app/release-key.jks Normal file

Binary file not shown.

View File

@@ -11,13 +11,13 @@
- DELETE /videos?profile_id=X&exclude_ids= — Videos des Profils loeschen (ausser lokal gespeicherte)
- POST /videos/{id}/download — Download triggern
- GET /videos/{id}/stream — Video streamen
- GET /videos/{id}/file — Video-Datei zum Download ausliefern
- GET /videos/{id}/file — Video-Datei zum Download ausliefern, setzt Download-Status zurueck wenn Datei fehlt
- DELETE /videos/{id}/file — Server-Datei loeschen (nach lokalem Download)
- WS /ws — WebSocket, benachrichtigt Clients bei neuen Videos
## Services
- VideoService — Videos speichern, abrufen, Status verwalten
- DownloadService — yt-dlp aufrufen, Video herunterladen
- StreamService — Live-Streaming via yt-dlp + ffmpeg
- StreamService — ffmpeg Live-Muxing von Video+Audio, gleichzeitiges Streaming und Speichern
## Model
- Video — id, title, youtuber, thumbnail_url, youtube_url, file_path
- Profile — id, name
@@ -27,11 +27,11 @@
- AllVideosScreen — alle Videos als Cards
- DownloadedVideosScreen — heruntergeladene Videos als Cards
- VideoDetailScreen — Starten, Download, Zurück, Löschen
- VideoPlayerScreen — Player mit Standard-Controls
- VideoPlayerScreen — Player mit Standard-Controls, Rotation behaelt Video-Position bei
## ViewModel
- VideoViewModel — Video-State verwalten, API-Aufrufe triggern, lokal bevorzugen sonst streamen
## Services
- LocalStorageService — Videos lokal speichern, pruefen, loeschen
- LocalStorageService — Videos und Metadaten lokal speichern, pruefen, loeschen (offline verfuegbar)
## API
- ServerApi — kommuniziert mit FastAPI (GET /profiles, GET /videos, POST /download, GET /stream, GET /videos/{id}/file, DELETE /videos/{id}/file, DELETE /videos)
## Model

View File

@@ -1,6 +1,7 @@
FROM python:3.12-slim
RUN apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y ffmpeg curl unzip && rm -rf /var/lib/apt/lists/* \
&& curl -fsSL https://deno.land/install.sh | DENO_INSTALL=/usr/local sh
WORKDIR /app
@@ -9,4 +10,4 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
CMD ["uvicorn", "base.app:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

View File

@@ -0,0 +1,38 @@
from fastapi import APIRouter
from api.schemas import CleanupRequest, VideoCreate, VideoResponse
from database.database import DbSession
from model.profile import Profile
from model.video import Video
from notify.notify_clients import notifyClients
router = APIRouter()
@router.get("/profiles")
def getAll(db: DbSession):
return Profile.getAll(db)
@router.post("/profiles/{profileId}/videos", status_code=204)
async def createVideo(profileId: int, videoData: VideoCreate, db: DbSession):
title = videoData.title
youtuber = videoData.youtuber
thumbnailUrl = videoData.thumbnailUrl
youtubeUrl = videoData.youtubeUrl
Video.deleteIfExists(db, youtubeUrl, profileId)
Video.create(db, title, youtuber, thumbnailUrl, youtubeUrl, profileId)
await notifyClients(profileId)
@router.get("/profiles/{profileId}/videos", response_model=list[VideoResponse])
def getVideos(profileId: int, db: DbSession):
return Video.getAll(db, profileId=profileId)
@router.post("/profiles/{profileId}/videos/cleanup")
def cleanupVideos(profileId: int, request: CleanupRequest, db: DbSession):
excludeIds = request.excludeIds
count = Video.deleteNotDownloaded(db, profileId, excludeIds)
return {"deleted": count}

24
backend/api/schemas.py Normal file
View File

@@ -0,0 +1,24 @@
from pydantic import BaseModel, ConfigDict
class VideoCreate(BaseModel):
title: str
youtuber: str
thumbnailUrl: str
youtubeUrl: str
class VideoResponse(BaseModel):
id: int
title: str
youtuber: str
thumbnailUrl: str
youtubeUrl: str
isDownloaded: bool
profileIds: list[int]
model_config = ConfigDict(from_attributes=True)
class CleanupRequest(BaseModel):
excludeIds: list[int] = []

View File

@@ -0,0 +1,53 @@
from pathlib import Path
from fastapi import APIRouter, HTTPException
from fastapi.responses import FileResponse, StreamingResponse
from database.database import DbSession
from download.download_service import downloadAsync
from model.video import Video
from stream.stream_service import streamAndSave
router = APIRouter(prefix="/videos")
@router.post("/{videoId}/download")
def download(videoId: int, db: DbSession):
video = Video.getById(db, videoId)
if not video:
raise HTTPException(404, "Video nicht gefunden")
if video.filePath:
return {"status": "already_downloaded"}
downloadAsync(videoId, video.youtubeUrl)
return {"status": "download_started"}
@router.get("/{videoId}/stream")
def stream(videoId: int, db: DbSession):
video = Video.getById(db, videoId)
if not video:
raise HTTPException(404, "Video nicht gefunden")
if not video.filePath:
return StreamingResponse(streamAndSave(videoId, video.youtubeUrl), media_type="video/mp4")
path = Path(video.filePath)
if not path.exists():
raise HTTPException(404, "Videodatei nicht gefunden")
return FileResponse(path, media_type="video/mp4")
@router.get("/{videoId}/file")
def getFile(videoId: int, db: DbSession):
path, video = Video.getValidPath(db, videoId)
if not path:
raise HTTPException(404, "Video noch nicht heruntergeladen")
return FileResponse(path, media_type="video/mp4", filename=f"{video.title}.mp4")
@router.delete("/{videoId}/file")
def deleteFile(videoId: int, db: DbSession):
if not Video.deleteServerFile(db, videoId):
raise HTTPException(404, "Video nicht gefunden")
return {"status": "deleted"}

36
backend/base/app.py Normal file
View File

@@ -0,0 +1,36 @@
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from api.profile_controller import router as profilesRouter
from api.video_controller import router as videosRouter
from database.database import SessionLocal, createTables
from model.profile import Profile
from notify.notify_clients import registerWebsocket
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(videosRouter)
app.include_router(profilesRouter)
registerWebsocket(app)
@app.on_event("startup")
def startup():
createTables()
db = SessionLocal()
if db.query(Profile).count() == 0:
db.add(Profile(name="Standard"))
db.commit()
db.close()
@app.get("/")
def root():
return {"status": "running"}

View File

View File

@@ -1,5 +1,8 @@
from typing import Annotated
from fastapi import Depends
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, declarative_base
from sqlalchemy.orm import Session, sessionmaker, declarative_base
DATABASE_URL = "sqlite:///videos/youtubeapp.db"
@@ -8,13 +11,16 @@ SessionLocal = sessionmaker(bind=engine)
Base = declarative_base()
def create_tables():
def createTables():
Base.metadata.create_all(bind=engine)
def get_db():
def getDb():
db = SessionLocal()
try:
yield db
finally:
db.close()
DbSession = Annotated[Session, Depends(getDb)]

View File

View File

@@ -0,0 +1,40 @@
import subprocess
import threading
from pathlib import Path
from database.database import SessionLocal
from model.video import Video
VIDEOS_DIR = "/videos"
MIN_VALID_SIZE = 1024 * 100 # 100 KB
def downloadAsync(videoId: int, youtubeUrl: str):
threading.Thread(target=downloadVideo, args=(videoId, youtubeUrl)).start()
def downloadVideo(videoId: int, youtubeUrl: str):
outputPath = f"{VIDEOS_DIR}/{videoId}.mp4"
subprocess.run(
[
"yt-dlp",
"-f", "bestvideo[ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]",
"-o", outputPath,
"--merge-output-format", "mp4",
"--force-overwrites",
"--no-continue",
youtubeUrl,
],
check=True,
)
path = Path(outputPath)
if not path.exists() or path.stat().st_size < MIN_VALID_SIZE:
return
db = SessionLocal()
try:
Video.updateFilePath(db, videoId, outputPath)
finally:
db.close()

View File

@@ -1,59 +0,0 @@
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from database import SessionLocal, create_tables
from models import Profile
from routes.videos import profiles_router, router as videos_router
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(videos_router)
app.include_router(profiles_router)
# --- WebSocket ---
connected_clients: set[WebSocket] = set()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
connected_clients.add(websocket)
try:
while True:
await websocket.receive_text()
except WebSocketDisconnect:
connected_clients.discard(websocket)
async def notify_clients(profile_ids: list[int]):
message = ",".join(str(pid) for pid in profile_ids)
for client in list(connected_clients):
try:
await client.send_text(message)
except Exception:
connected_clients.discard(client)
# --- Startup ---
@app.on_event("startup")
def startup():
create_tables()
db = SessionLocal()
if db.query(Profile).count() == 0:
db.add(Profile(name="Standard"))
db.commit()
db.close()
@app.get("/")
def root():
return {"status": "running"}

View File

16
backend/model/profile.py Normal file
View File

@@ -0,0 +1,16 @@
from sqlalchemy import Column, Integer, String
from sqlalchemy.orm import Session
from database.database import Base
class Profile(Base):
__tablename__ = "profiles"
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String, nullable=False, unique=True)
@classmethod
def getAll(cls, db: Session) -> list[dict]:
profiles = db.query(cls).all()
return [{"id": p.id, "name": p.name} for p in profiles]

View File

@@ -0,0 +1,10 @@
from sqlalchemy import Column, ForeignKey, Integer, Table
from database.database import Base
videoProfiles = Table(
"video_profiles",
Base.metadata,
Column("video_id", Integer, ForeignKey("videos.id", ondelete="CASCADE")),
Column("profile_id", Integer, ForeignKey("profiles.id", ondelete="CASCADE")),
)

123
backend/model/video.py Normal file
View File

@@ -0,0 +1,123 @@
from pathlib import Path
from sqlalchemy import Column, Integer, String
from sqlalchemy.orm import Session, relationship
from database.database import Base
from model.profile import Profile
from model.profile_video import videoProfiles
class Video(Base):
__tablename__ = "videos"
id = Column(Integer, primary_key=True, autoincrement=True)
title = Column(String, nullable=False)
youtuber = Column(String, nullable=False)
thumbnailUrl = Column("thumbnail_url", String, nullable=False)
youtubeUrl = Column("youtube_url", String, nullable=False)
filePath = Column("file_path", String, nullable=True)
profiles = relationship("Profile", secondary=videoProfiles, backref="videos")
@property
def isDownloaded(self) -> bool:
return self.filePath is not None
@property
def profileIds(self) -> list[int]:
return [p.id for p in self.profiles]
@classmethod
def deleteIfExists(cls, db: Session, youtubeUrl: str, profileId: int | None):
if not profileId:
profileId = 1
videos = db.query(cls).filter(
cls.youtubeUrl == youtubeUrl,
cls.profiles.any(Profile.id == profileId),
).all()
for video in videos:
db.delete(video)
db.commit()
@classmethod
def create(
cls,
db: Session,
title: str,
youtuber: str,
thumbnailUrl: str,
youtubeUrl: str,
profileId: int | None,
) -> "Video":
if not profileId:
profileId = 1
video = cls(
title=title,
youtuber=youtuber,
thumbnailUrl=thumbnailUrl,
youtubeUrl=youtubeUrl,
)
profile = db.query(Profile).filter(Profile.id == profileId).first()
if profile:
video.profiles.append(profile)
db.add(video)
db.commit()
db.refresh(video)
return video
@classmethod
def getAll(cls, db: Session, profileId: int | None = None) -> list["Video"]:
query = db.query(cls)
if profileId:
query = query.filter(cls.profiles.any(Profile.id == profileId))
return query.order_by(cls.id.desc()).all()
@classmethod
def getById(cls, db: Session, videoId: int) -> "Video | None":
return db.query(cls).filter(cls.id == videoId).first()
@classmethod
def updateFilePath(cls, db: Session, videoId: int, path: str | None):
video = cls.getById(db, videoId)
if video:
video.filePath = path
db.commit()
@classmethod
def getValidPath(cls, db: Session, videoId: int):
video = cls.getById(db, videoId)
if not video or not video.filePath:
return None, None
path = Path(video.filePath)
if not path.exists():
cls.updateFilePath(db, videoId, None)
return None, None
return path, video
@classmethod
def deleteServerFile(cls, db: Session, videoId: int) -> bool:
video = cls.getById(db, videoId)
if not video:
return False
if video.filePath:
path = Path(video.filePath)
if path.exists():
path.unlink()
cls.updateFilePath(db, videoId, None)
return True
@classmethod
def deleteNotDownloaded(cls, db: Session, profileId: int, excludeIds: list[int] | None = None) -> int:
query = db.query(cls).filter(
cls.profiles.any(Profile.id == profileId),
)
if excludeIds:
query = query.filter(cls.id.notin_(excludeIds))
videos = query.all()
videoIds = [v.id for v in videos]
if not videoIds:
return 0
db.execute(videoProfiles.delete().where(videoProfiles.c.video_id.in_(videoIds)))
db.query(cls).filter(cls.id.in_(videoIds)).delete(synchronize_session=False)
db.commit()
return len(videoIds)

View File

@@ -1,30 +0,0 @@
from sqlalchemy import Column, ForeignKey, Integer, String, Table
from sqlalchemy.orm import relationship
from database import Base
video_profiles = Table(
"video_profiles",
Base.metadata,
Column("video_id", Integer, ForeignKey("videos.id", ondelete="CASCADE")),
Column("profile_id", Integer, ForeignKey("profiles.id", ondelete="CASCADE")),
)
class Profile(Base):
__tablename__ = "profiles"
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String, nullable=False, unique=True)
class Video(Base):
__tablename__ = "videos"
id = Column(Integer, primary_key=True, autoincrement=True)
title = Column(String, nullable=False)
youtuber = Column(String, nullable=False)
thumbnail_url = Column(String, nullable=False)
youtube_url = Column(String, nullable=False)
file_path = Column(String, nullable=True)
profiles = relationship("Profile", secondary=video_profiles, backref="videos")

View File

View File

@@ -0,0 +1,26 @@
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
connectedClients: set[WebSocket] = set()
async def notifyClients(profileId: int | None):
if not profileId:
profileId = 1
message = str(profileId)
for client in list(connectedClients):
try:
await client.send_text(message)
except Exception:
connectedClients.discard(client)
def registerWebsocket(app: FastAPI):
@app.websocket("/ws")
async def websocketEndpoint(websocket: WebSocket):
await websocket.accept()
connectedClients.add(websocket)
try:
while True:
await websocket.receive_text()
except WebSocketDisconnect:
connectedClients.discard(websocket)

View File

@@ -1,131 +0,0 @@
import asyncio
import threading
from pathlib import Path
from typing import Optional
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query
from fastapi.responses import FileResponse, StreamingResponse
from sqlalchemy.orm import Session
from database import get_db
from schemas import CleanupRequest, ProfileResponse, VideoCreate, VideoResponse
from services import video_service
from services.download_service import download_video
from services.stream_service import stream_video_live
from services.video_service import update_file_path
router = APIRouter(prefix="/videos", tags=["videos"])
@router.post("", response_model=list[VideoResponse])
async def create_videos(videos_data: list[VideoCreate], db: Session = Depends(get_db)):
created_ids = []
profile_ids = set()
for video_data in reversed(videos_data):
video_id_match = video_data.youtube_url.split("v=")[-1].split("&")[0]
video_service.delete_by_youtube_id(db, video_id_match)
video = video_service.create_video(db, video_data)
created_ids.append(video.id)
if video_data.profile_id:
profile_ids.add(video_data.profile_id)
videos = [video_service.get_video(db, vid) for vid in created_ids]
if profile_ids:
from main import notify_clients
await notify_clients(list(profile_ids))
return [VideoResponse.from_model(v) for v in videos if v]
@router.get("", response_model=list[VideoResponse])
def get_all_videos(profile_id: Optional[int] = Query(None), db: Session = Depends(get_db)):
videos = video_service.get_all_videos(db, profile_id=profile_id)
return [VideoResponse.from_model(v) for v in videos]
@router.get("/downloaded", response_model=list[VideoResponse])
def get_downloaded_videos(profile_id: Optional[int] = Query(None), db: Session = Depends(get_db)):
videos = video_service.get_downloaded_videos(db, profile_id=profile_id)
return [VideoResponse.from_model(v) for v in videos]
@router.post("/cleanup")
def cleanup_videos(request: CleanupRequest, db: Session = Depends(get_db)):
count = video_service.delete_not_downloaded(db, request.profile_id, request.exclude_ids or None)
return {"deleted": count}
@router.post("/{video_id}/download")
def trigger_download(video_id: int, db: Session = Depends(get_db)):
video = video_service.get_video(db, video_id)
if not video:
raise HTTPException(status_code=404, detail="Video nicht gefunden")
if video.file_path:
return {"status": "already_downloaded"}
thread = threading.Thread(target=download_video, args=(video.id, video.youtube_url))
thread.start()
return {"status": "download_started"}
@router.get("/{video_id}/stream")
def stream_video(video_id: int, db: Session = Depends(get_db)):
video = video_service.get_video(db, video_id)
if not video:
raise HTTPException(status_code=404, detail="Video nicht gefunden")
if not video.file_path:
def stream_and_save():
output_path = f"/videos/{video_id}.mp4"
yield from stream_video_live(video_id, video.youtube_url)
if Path(output_path).exists():
sdb = __import__("database").SessionLocal()
try:
update_file_path(sdb, video_id, output_path)
finally:
sdb.close()
return StreamingResponse(stream_and_save(), media_type="video/mp4")
path = Path(video.file_path)
if not path.exists():
raise HTTPException(status_code=404, detail="Videodatei nicht gefunden")
return FileResponse(path, media_type="video/mp4")
@router.get("/{video_id}/file")
def download_file(video_id: int, db: Session = Depends(get_db)):
video = video_service.get_video(db, video_id)
if not video:
raise HTTPException(status_code=404, detail="Video nicht gefunden")
if not video.file_path:
raise HTTPException(status_code=404, detail="Video noch nicht heruntergeladen")
path = Path(video.file_path)
if not path.exists():
video_service.update_file_path(db, video_id, None)
raise HTTPException(status_code=404, detail="Video noch nicht heruntergeladen")
return FileResponse(path, media_type="video/mp4", filename=f"{video.title}.mp4")
@router.delete("/{video_id}/file")
def delete_server_file(video_id: int, db: Session = Depends(get_db)):
video = video_service.get_video(db, video_id)
if not video:
raise HTTPException(status_code=404, detail="Video nicht gefunden")
if video.file_path:
path = Path(video.file_path)
if path.exists():
path.unlink()
video_service.update_file_path(db, video_id, None)
return {"status": "deleted"}
profiles_router = APIRouter(prefix="/profiles", tags=["profiles"])
@profiles_router.get("", response_model=list[ProfileResponse])
def get_profiles(db: Session = Depends(get_db)):
return video_service.get_all_profiles(db)

View File

@@ -1,47 +0,0 @@
from pydantic import BaseModel
class VideoCreate(BaseModel):
title: str
youtuber: str
thumbnail_url: str
youtube_url: str
profile_id: int | None = None
class VideoResponse(BaseModel):
id: int
title: str
youtuber: str
thumbnail_url: str
youtube_url: str
is_downloaded: bool
profile_ids: list[int]
class Config:
from_attributes = True
@classmethod
def from_model(cls, video):
return cls(
id=video.id,
title=video.title,
youtuber=video.youtuber,
thumbnail_url=video.thumbnail_url,
youtube_url=video.youtube_url,
is_downloaded=video.file_path is not None,
profile_ids=[p.id for p in video.profiles],
)
class CleanupRequest(BaseModel):
profile_id: int
exclude_ids: list[int] = []
class ProfileResponse(BaseModel):
id: int
name: str
class Config:
from_attributes = True

View File

@@ -1,27 +0,0 @@
import subprocess
from database import SessionLocal
from services.video_service import get_video, update_file_path
VIDEOS_DIR = "/videos"
def download_video(video_id: int, youtube_url: str):
output_path = f"{VIDEOS_DIR}/{video_id}.mp4"
subprocess.run(
[
"yt-dlp",
"-f", "bestvideo[ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]",
"-o", output_path,
"--merge-output-format", "mp4",
youtube_url,
],
check=True,
)
db = SessionLocal()
try:
update_file_path(db, video_id, output_path)
finally:
db.close()

View File

@@ -1,69 +0,0 @@
from sqlalchemy.orm import Session
from models import Profile, Video, video_profiles
from schemas import VideoCreate
def create_video(db: Session, video_data: VideoCreate) -> Video:
profile_id = video_data.profile_id
data = video_data.model_dump(exclude={"profile_id"})
video = Video(**data)
if not profile_id:
profile_id = 1
profile = db.query(Profile).filter(Profile.id == profile_id).first()
if profile:
video.profiles.append(profile)
db.add(video)
db.commit()
db.refresh(video)
return video
def get_all_videos(db: Session, profile_id: int | None = None) -> list[Video]:
query = db.query(Video)
if profile_id:
query = query.filter(Video.profiles.any(Profile.id == profile_id))
return query.order_by(Video.id.desc()).all()
def get_downloaded_videos(db: Session, profile_id: int | None = None) -> list[Video]:
query = db.query(Video).filter(Video.file_path.isnot(None))
if profile_id:
query = query.filter(Video.profiles.any(Profile.id == profile_id))
return query.order_by(Video.id.desc()).all()
def get_video(db: Session, video_id: int) -> Video | None:
return db.query(Video).filter(Video.id == video_id).first()
def delete_by_youtube_id(db: Session, youtube_id: str):
db.query(Video).filter(Video.youtube_url.contains(youtube_id)).delete(synchronize_session=False)
db.commit()
def update_file_path(db: Session, video_id: int, path: str):
video = get_video(db, video_id)
if video:
video.file_path = path
db.commit()
def delete_not_downloaded(db: Session, profile_id: int, exclude_ids: list[int] | None = None) -> int:
query = db.query(Video).filter(
Video.profiles.any(Profile.id == profile_id),
)
if exclude_ids:
query = query.filter(Video.id.notin_(exclude_ids))
videos = query.all()
video_ids = [v.id for v in videos]
if not video_ids:
return 0
db.execute(video_profiles.delete().where(video_profiles.c.video_id.in_(video_ids)))
db.query(Video).filter(Video.id.in_(video_ids)).delete(synchronize_session=False)
db.commit()
return len(video_ids)
def get_all_profiles(db: Session) -> list[Profile]:
return db.query(Profile).all()

View File

View File

@@ -1,17 +1,31 @@
import subprocess
from pathlib import Path
from database.database import SessionLocal
VIDEOS_DIR = "/videos"
CHUNK_SIZE = 64 * 1024
def _get_stream_urls(youtube_url: str):
def streamAndSave(videoId: int, youtubeUrl: str):
from model.video import Video # Lazy-Import gegen Zirkular
outputPath = f"{VIDEOS_DIR}/{videoId}.mp4"
yield from streamVideoLive(videoId, youtubeUrl)
if Path(outputPath).exists():
db = SessionLocal()
try:
Video.updateFilePath(db, videoId, outputPath)
finally:
db.close()
def _getStreamUrls(youtubeUrl: str):
result = subprocess.run(
[
"yt-dlp",
"-f", "bestvideo[ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]",
"--print", "urls",
youtube_url,
youtubeUrl,
],
capture_output=True, text=True, timeout=30,
)
@@ -26,24 +40,24 @@ def _get_stream_urls(youtube_url: str):
return None, None
def stream_video_live(video_id: int, youtube_url: str):
output_path = f"{VIDEOS_DIR}/{video_id}.mp4"
def streamVideoLive(videoId: int, youtubeUrl: str):
outputPath = f"{VIDEOS_DIR}/{videoId}.mp4"
video_url, audio_url = _get_stream_urls(youtube_url)
if not video_url:
videoUrl, audioUrl = _getStreamUrls(youtubeUrl)
if not videoUrl:
return
if audio_url:
if audioUrl:
cmd = [
"ffmpeg",
"-reconnect", "1",
"-reconnect_streamed", "1",
"-reconnect_delay_max", "5",
"-i", video_url,
"-i", videoUrl,
"-reconnect", "1",
"-reconnect_streamed", "1",
"-reconnect_delay_max", "5",
"-i", audio_url,
"-i", audioUrl,
"-c:v", "copy",
"-c:a", "aac",
"-movflags", "frag_keyframe+empty_moov+default_base_moof",
@@ -56,7 +70,7 @@ def stream_video_live(video_id: int, youtube_url: str):
"-reconnect", "1",
"-reconnect_streamed", "1",
"-reconnect_delay_max", "5",
"-i", video_url,
"-i", videoUrl,
"-c", "copy",
"-movflags", "frag_keyframe+empty_moov+default_base_moof",
"-f", "mp4",
@@ -70,7 +84,7 @@ def stream_video_live(video_id: int, youtube_url: str):
)
try:
with open(output_path, "wb") as f:
with open(outputPath, "wb") as f:
while True:
chunk = process.stdout.read(CHUNK_SIZE)
if not chunk:
@@ -86,6 +100,6 @@ def stream_video_live(video_id: int, youtube_url: str):
if process.stdout:
process.stdout.close()
path = Path(output_path)
path = Path(outputPath)
if process.returncode != 0 and path.exists():
path.unlink()

View File

@@ -0,0 +1,9 @@
const SERVER_BASE = "http://marha.local:8000";
browser.runtime.onMessage.addListener(({ profileId, video }) => {
fetch(`${SERVER_BASE}/profiles/${profileId}/videos`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(video),
}).catch(() => {});
});

View File

@@ -1,9 +0,0 @@
const SERVER_URL = "http://marha.local:8000/videos";
browser.runtime.onMessage.addListener((videos) => {
fetch(SERVER_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(videos),
}).catch(() => {});
});

View File

@@ -11,14 +11,14 @@
"content_scripts": [
{
"matches": ["*://www.youtube.com/*"],
"js": ["content.js"]
"js": ["tracking/content.js"]
}
],
"background": {
"scripts": ["background.js"]
"scripts": ["api/background.js"]
},
"browser_action": {
"default_popup": "popup.html",
"default_popup": "config/popup.html",
"default_title": "Profil auswählen"
}
}

View File

@@ -18,51 +18,28 @@ function extractVideoFromCard(element) {
return {
title,
youtuber,
thumbnail_url: thumbnail || `https://img.youtube.com/vi/${match[1]}/hqdefault.jpg`,
youtube_url: `https://www.youtube.com/watch?v=${match[1]}`,
thumbnailUrl: thumbnail || `https://img.youtube.com/vi/${match[1]}/hqdefault.jpg`,
youtubeUrl: `https://www.youtube.com/watch?v=${match[1]}`,
};
}
function collectVideos(elements) {
const videos = [];
for (const el of elements) {
const video = extractVideoFromCard(el);
if (!video) continue;
if (sentUrls.has(video.youtube_url)) continue;
sentUrls.add(video.youtube_url);
videos.push(video);
}
return videos;
}
// --- Debounced Batch-Versand ---
let pendingVideos = [];
let sendTimer = null;
async function queueVideos(videos) {
pendingVideos.push(...videos);
if (!sendTimer) {
sendTimer = setTimeout(async () => {
if (pendingVideos.length > 0) {
async function sendVideo(video) {
const stored = await browser.storage.local.get("profileId");
const profileId = stored.profileId || null;
const batch = pendingVideos.map((v) => ({ ...v, profile_id: profileId }));
console.log(`[YT-Erfasser] ${batch.length} Videos senden (Profil: ${profileId})`);
browser.runtime.sendMessage(batch);
}
pendingVideos = [];
sendTimer = null;
}, 250);
}
const profileId = stored.profileId || 1;
console.log(`[YT-Erfasser] Video senden (Profil: ${profileId})`, video.title);
browser.runtime.sendMessage({ profileId, video });
}
// --- IntersectionObserver: nur sichtbare Cards erfassen ---
const visibilityObserver = new IntersectionObserver((entries) => {
const cards = entries.filter((e) => e.isIntersecting).map((e) => e.target);
if (cards.length > 0) {
queueVideos(collectVideos(cards));
for (const entry of entries) {
if (!entry.isIntersecting) continue;
const video = extractVideoFromCard(entry.target);
if (!video) continue;
if (sentUrls.has(video.youtubeUrl)) continue;
sentUrls.add(video.youtubeUrl);
sendVideo(video);
}
}, { threshold: 0.5 });

6
database.md Normal file
View File

@@ -0,0 +1,6 @@
# Backend
Profile(id, name) - Profiledata
Video(id, title, youtuber, imgUrl, videoUrl, profile) - Videodata for profiles
# App
Profile(id, name) - Profiledata, sync with backend
Video(id, title, youtuber, imgPath, videoPath, profile) - Videodata for downloaded videos

View File

@@ -1,16 +1,38 @@
# Aufgaben
## Browser
- Sichtbare Youtube Videos werden erfasst
- Videodaten (Titel, Youtuber, Bild, Url) werden nach dem erfassen gruppiert an den Server gesendet
- Einstellung: Profil auswählen
## App
- Ansicht: Navigation mit Alle Videos, Heruntergeladen
- Startansicht:
- Navigation unten: Alle Videos, Heruntergeladen
- Icons oben rechts: Mülleimer (Löschen), Benutzer (Profile)
- Alle Videos: Videos als Cards auflisten (Untereinander: Bild, Youtuber, Titel)
- Heruntergeladen: Heruntergeladene Videos als Cards auflisten (Untereinander: Bild, Youtuber, Titel)
- Klick auf Card zeigt die Videoübersicht (Starten, Download, Zurück)
- Klick auf Startet startet den Stream über den Server mit den Standard Video-Controls und einem Zurück-Button
- Klick auf Download lädt das Video herunter und wird lokal auf dem Client gespeichert
- Klick auf Icon zeigt verfügbare Profile
- Das ausgewählte Profil wird persistiert und bestimmt welche Videos angezeigt werden
- Klick auf Icon löscht alle nicht heruntergeladenen Videos vom aktuellen Profil
- Bei neuen Videoeinträgen in der DB werden die Videos für das zugehörige Profil in der App aktualisiert
- Klick auf ein Video zeigt die Videoübersicht
- Neue Videoeinträge in der DB aktualisieren direkt die Videoliste
- Funktioniert nur Online und wenn Server verfügbar
- Heruntergeladen: Heruntergeladene (lokale) Videos als Cards auflisten (Untereinander: Bild, Youtuber, Titel)
- Videodaten und Videodatei werden lokal gespeichert (funktionieren Offline)
- Mülleimer Icon: Alle Videodaten von nicht heruntergeladenen Videos zum aktuellen Profil löschen
- Benutzer Icon: Verfügbare Profile anzeigen
- Klick auf ein Profile setzt dieses als das aktuelle Profil
- Es werden nur Videos zu dem Profil angezeigt
- Videos ohne Profilzuweisung werden automatisch dem Standardprofil zugeordnet
- Videoübersicht:
- Oben links: Zurück-Button
- Unter Zurück-Button: Thumbnail
- Unten: Abspielen und Download Buttons
- Abspielen:
- "Zurück"-Button oben links
- Standard Videos Controls
- Startet einen Stream über den Server
- Download:
- Video auf dem Server herunterladen, Video lokal speichern, Video auf dem Server löschen
- Ladeanimation währenddessen
- Beim Abspielen wird das heruntergeladene Video priorisiert
- Fehlerbehandlung
- Server nicht erreichbar: Fehlermeldung "Server nicht erreichbar" anzeigen unter Alle Videos

32
modules.md Normal file
View File

@@ -0,0 +1,32 @@
# Browser Extension
## Tracking Module
- content.js - Scan visable youtube videos
## Api Module
- background.js - Send scanned videos to server
## Config Module
- popup.js - Select Profile
- popup.html - Template for popup
## Base Module
- manifest.json - Register extension
# Backend
## Database Module
- database.py - Setup database
## Model Module
- profile.py - add profile columns and model methods
- video.py - add video columns and model methods
- profile_video.py - table for profile-video relation
## Api Module
- video_controller.py - Video routes
- profile_controller.py - Profile routes
- schemas.py - Validate API
## Download Module
- download_service.py - Handle downloads
## Stream Module
- stream_service.py - Handle streaming
## Notify Module
- notify_clients.py - Websocket, update video list
## Base Module
- app.py - App start
# App

View File

@@ -2,7 +2,7 @@
- Javascript: Daten angezeigter Videos an Server senden
# Server
- FastAPI: Videodaten empfangen
- yt-dlp + ffmpeg: Video herunterladen, Videos streamen
- yt-dlp + ffmpeg + Deno: Video herunterladen, Videos streamen
- SQLite: Daten persistieren
# App
- Kotlin: Videos auflisten, Download triggern, Videos abspielen