This commit is contained in:
Marek
2026-04-05 23:26:55 +02:00
parent b3b4410ee0
commit b6635a107d
21 changed files with 229 additions and 75 deletions

View File

@@ -31,41 +31,48 @@ Drei Komponenten:
### Server (`backend/`)
- **Python, FastAPI, SQLAlchemy, SQLite** (`videos/youtubeapp.db`)
- **yt-dlp + ffmpeg** fuer Video-Download und Live-Streaming
- **yt-dlp + ffmpeg** fuer Video-Download und Streaming
- **WebSocket** (`/ws`) — benachrichtigt verbundene Clients bei neuen Videos
- Dockerisiert: `docker compose up --build -d` im `backend/` Verzeichnis
- Laeuft auf `http://localhost:8000`
- Download-Service speichert Videos unter `/videos/{id}.mp4`
- Stream-Service: heruntergeladene Videos von Datei, sonst Live-Stream via yt-dlp + ffmpeg (fragmentiertes MP4)
- Stream-Service: heruntergeladene Videos von Datei, sonst progressiver Download via yt-dlp mit gleichzeitigem Streaming
- Dedup: beim Batch-Import wird bestehender Eintrag mit gleicher Video-ID geloescht und neu eingefuegt
- Sortierung: nach ID absteigend (erstes Video im Batch bekommt hoechste ID)
- Profile: fest in DB definiert, Videos ueber Many-to-Many zugeordnet
- Nach lokalem Download wird die Server-Datei geloescht (file_path auf null)
### App (`app/`)
- **Kotlin, Jetpack Compose**, Android/Android TV
- Gradle-Projekt, Modul `frontend`
- Screens: AllVideos (Grid), Downloaded, VideoDetail, VideoPlayer
- Screens: AllVideos (Grid), Downloaded (lokal verfuegbare Videos), VideoDetail, VideoPlayer
- Retrofit fuer API-Calls, Coil fuer Thumbnails, ExoPlayer fuer Streaming
- Navigation mit TopBar (Profil-Auswahl) und Bottom Bar, Dark Theme
- OkHttp WebSocket-Client — automatisches Neuladen bei neuen Videos
- Navigation mit TopBar (Profil-Auswahl, Aufraeumen-Icon) und Bottom Bar, Dark Theme
- Profil-Auswahl wird in SharedPreferences persistiert, filtert Videos nach Profil
- Lokaler Download: Videos werden auf dem Geraet gespeichert, lokal bevorzugt abgespielt
- Aufraeumen: loescht alle nicht lokal gespeicherten Videos des Profils (sendet lokale IDs als Ausnahme)
- Server-IP konfigurierbar in `ApiClient.kt` (aktuell `192.168.178.92`)
- Emulator: Android Studio → Device Manager → Pixel 6a, API 35
## API Endpoints
- `GET /profiles` — alle Profile abrufen
- `POST /videos` — Video-Batch von Extension empfangen (Dedup, Reverse-Insert, Profil-Zuordnung)
- `POST /videos` — Video-Batch von Extension empfangen (Dedup, Reverse-Insert, Profil-Zuordnung, WebSocket-Benachrichtigung)
- `GET /videos` — alle Videos abrufen (optional `?profile_id=X`, sortiert nach ID absteigend)
- `GET /videos/downloaded` — heruntergeladene Videos abrufen (optional `?profile_id=X`)
- `DELETE /videos?profile_id=X&exclude_ids=` — Videos des Profils loeschen (ausser lokal gespeicherte)
- `POST /videos/{id}/download` — Download auf Server triggern
- `GET /videos/{id}/stream` — Video streamen (von Datei oder Live via yt-dlp/ffmpeg)
- `GET /videos/{id}/stream` — Video streamen (von Datei oder progressiver Download via yt-dlp)
- `GET /videos/{id}/file` — Video-Datei zum Download auf Client ausliefern
- `DELETE /videos/{id}/file` — Server-Datei loeschen (nach lokalem Download)
- `WS /ws` — WebSocket, sendet Profile-IDs bei neuen Videos
## Projektstruktur
```
backend/
main.py — FastAPI App, CORS, Startup, Seed-Profile
main.py — FastAPI App, CORS, Startup, Seed-Profile, WebSocket
database.py — SQLAlchemy Engine, Session, Base
models.py — Video, Profile, video_profiles (Many-to-Many)
schemas.py — Pydantic Schemas (VideoCreate, VideoResponse, ProfileResponse)
@@ -73,14 +80,14 @@ backend/
services/
video_service.py — CRUD-Operationen, Dedup, Profil-Filter
download_service.py — yt-dlp Download
stream_service.py — Live-Streaming via yt-dlp + ffmpeg
stream_service.py — Progressiver Download + Streaming via yt-dlp
Dockerfile — Python 3.12 + ffmpeg
docker-compose.yml — Service-Definition, Port 8000, Volume /videos
.dockerignore — videos/, __pycache__/
.gitignore — videos/, __pycache__/
browser_extension/
manifest.json — Manifest V2, Permissions, browser_action
manifest.json — Manifest V2, Permissions, browser_action, storage
content.js — DOM-Extraktion + IntersectionObserver + Batch-Versand mit Profil
background.js — Batch-POST an Server
popup.html — Profil-Auswahl UI
@@ -93,9 +100,12 @@ app/
data/ — Video, Profile, ApiClient, VideoApi, VideoRepository, LocalStorageService
ui/screens/ — AllVideos, Downloaded, VideoDetail, VideoPlayer
ui/components/ — VideoCard
ui/viewmodel/ — VideoViewModel
ui/viewmodel/ — VideoViewModel (inkl. WebSocket-Client)
ui/navigation/ — AppNavigation, Routes
ui/theme/ — Theme (Dark)
frontend/src/main/res/
layout/player_view.xml — PlayerView mit TextureView fuer TV-Kompatibilitaet
drawable/tv_banner.png — Android TV Launcher-Banner
```
## Entscheidungen
@@ -103,8 +113,9 @@ app/
- Kein Jellyfin — erfuellt nicht die Anforderung, Videos vor dem Download aufzulisten
- Kein PostgreSQL/MySQL — SQLite reicht fuer den Prototyp
- Profile fest in DB — kein UI zum Erstellen/Loeschen, werden direkt in der Datenbank verwaltet
- Videos werden auf dem Server gespeichert, Client speichert nur bei explizitem Download
- Server-Datei wird nach lokalem Download geloescht — spart Speicherplatz auf dem Server
- DOM-Extraktion statt ytInitialData-Parsing — funktioniert auch bei SPA-Navigation und Scrollen
- IntersectionObserver statt blindem Scan — nur sichtbare Videos erfassen
- Live-Streaming via yt-dlp/ffmpeg statt synchronem Download vor dem Streamen
- Progressiver Download via yt-dlp mit gleichzeitigem Streaming statt komplettem Download vor dem Abspielen
- WebSocket statt Polling — effiziente Echtzeit-Aktualisierung der Videoliste
- Sprache der Dokumentation: Deutsch

View File

@@ -24,6 +24,13 @@ class LocalStorageService(private val context: Context) {
fun deleteLocalFile(videoId: Int): Boolean = videoFile(videoId).delete()
fun getLocalVideoIds(): List<Int> {
return videosDir().listFiles()
?.filter { it.extension == "mp4" }
?.mapNotNull { it.nameWithoutExtension.toIntOrNull() }
?: emptyList()
}
fun downloadAndSave(videoId: Int): File {
val url = "${ApiClient.BASE_URL}videos/$videoId/file"
val file = videoFile(videoId)

View File

@@ -15,8 +15,11 @@ interface VideoApi {
@POST("videos/{id}/download")
suspend fun triggerDownload(@Path("id") id: Int): Map<String, String>
@retrofit2.http.DELETE("videos")
suspend fun deleteNotDownloaded(@Query("profile_id") profileId: Int): Map<String, Int>
@POST("videos/cleanup")
suspend fun cleanupVideos(@retrofit2.http.Body body: Map<String, @JvmSuppressWildcards Any>): Map<String, Int>
@retrofit2.http.DELETE("videos/{id}/file")
suspend fun deleteServerFile(@Path("id") id: Int): Map<String, String>
@GET("profiles")
suspend fun getProfiles(): List<Profile>

View File

@@ -13,8 +13,14 @@ class VideoRepository(private val api: VideoApi = ApiClient.api) {
response["status"] ?: "unknown"
}
suspend fun deleteNotDownloaded(profileId: Int): Result<Int> = runCatching {
val response = api.deleteNotDownloaded(profileId)
suspend fun deleteServerFile(videoId: Int): Result<String> = runCatching {
val response = api.deleteServerFile(videoId)
response["status"] ?: "unknown"
}
suspend fun cleanupVideos(profileId: Int, excludeIds: List<Int>): Result<Int> = runCatching {
val body = mapOf("profile_id" to profileId, "exclude_ids" to excludeIds)
val response = api.cleanupVideos(body)
response["deleted"] ?: 0
}

View File

@@ -47,7 +47,7 @@ fun AllVideosScreen(viewModel: VideoViewModel, onVideoClick: (Int) -> Unit) {
}
else -> {
LazyVerticalGrid(
columns = GridCells.Adaptive(minSize = 250.dp),
columns = GridCells.Adaptive(minSize = 160.dp),
contentPadding = PaddingValues(8.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),

View File

@@ -52,7 +52,7 @@ fun DownloadedScreen(viewModel: VideoViewModel, onVideoClick: (Int) -> Unit) {
}
else -> {
LazyVerticalGrid(
columns = GridCells.Adaptive(minSize = 250.dp),
columns = GridCells.Adaptive(minSize = 160.dp),
contentPadding = PaddingValues(8.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),

View File

@@ -48,7 +48,7 @@ fun VideoDetailScreen(
) {
val state by viewModel.state.collectAsState()
val video = viewModel.getVideoById(videoId)
val isLocal = viewModel.isLocallyAvailable(videoId)
val isLocal = remember(state.downloadStatus) { viewModel.isLocallyAvailable(videoId) }
val snackbarHostState = remember { SnackbarHostState() }
LaunchedEffect(state.downloadStatus) {

View File

@@ -1,8 +1,10 @@
package com.youtubeapp.ui.viewmodel
import android.content.Context
import android.util.Log
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.youtubeapp.data.ApiClient
import com.youtubeapp.data.LocalStorageService
import com.youtubeapp.data.Profile
import com.youtubeapp.data.Video
@@ -12,6 +14,11 @@ import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.Response
import okhttp3.WebSocket
import okhttp3.WebSocketListener
data class VideoUiState(
val allVideos: List<Video> = emptyList(),
@@ -28,6 +35,7 @@ class VideoViewModel : ViewModel() {
private val repository = VideoRepository()
private var localStorage: LocalStorageService? = null
private var prefs: android.content.SharedPreferences? = null
private var webSocket: WebSocket? = null
private val _state = MutableStateFlow(VideoUiState())
val state: StateFlow<VideoUiState> = _state
@@ -42,6 +50,37 @@ class VideoViewModel : ViewModel() {
_state.value = _state.value.copy(selectedProfileId = savedId)
}
}
if (webSocket == null) {
connectWebSocket()
}
}
private fun connectWebSocket() {
val wsUrl = ApiClient.BASE_URL.replace("http://", "ws://") + "ws"
val request = Request.Builder().url(wsUrl).build()
val client = OkHttpClient()
webSocket = client.newWebSocket(request, object : WebSocketListener() {
override fun onMessage(webSocket: WebSocket, text: String) {
val profileIds = text.split(",").mapNotNull { it.trim().toIntOrNull() }
val selected = _state.value.selectedProfileId
if (selected != null && selected in profileIds) {
loadAllVideos()
}
}
override fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {
Log.w("VideoViewModel", "WebSocket Fehler, reconnect in 5s", t)
viewModelScope.launch(Dispatchers.IO) {
delay(5000)
connectWebSocket()
}
}
})
}
override fun onCleared() {
webSocket?.close(1000, null)
super.onCleared()
}
fun loadProfiles() {
@@ -79,9 +118,10 @@ class VideoViewModel : ViewModel() {
fun loadDownloadedVideos() {
viewModelScope.launch {
_state.value = _state.value.copy(isLoading = true, error = null)
repository.getDownloadedVideos(profileId = _state.value.selectedProfileId)
repository.getAllVideos(profileId = _state.value.selectedProfileId)
.onSuccess { videos ->
_state.value = _state.value.copy(downloadedVideos = videos, isLoading = false)
val local = videos.filter { localStorage?.isLocallyAvailable(it.id) == true }
_state.value = _state.value.copy(downloadedVideos = local, isLoading = false)
}
.onFailure { e ->
_state.value = _state.value.copy(error = e.message, isLoading = false)
@@ -113,6 +153,7 @@ class VideoViewModel : ViewModel() {
}
localStorage?.downloadAndSave(videoId)
repository.deleteServerFile(videoId)
_state.value = _state.value.copy(
isDownloading = false,
downloadStatus = "Lokal gespeichert"
@@ -128,8 +169,9 @@ class VideoViewModel : ViewModel() {
fun deleteNotDownloaded() {
val profileId = _state.value.selectedProfileId ?: return
val localIds = localStorage?.getLocalVideoIds() ?: emptyList()
viewModelScope.launch {
repository.deleteNotDownloaded(profileId)
repository.cleanupVideos(profileId, localIds)
loadAllVideos()
}
}

View File

@@ -8,10 +8,12 @@
- GET /videos — alle Videos abrufen (optional nach Profil filtern)
- GET /profiles — alle Profile abrufen
- GET /videos/downloaded — heruntergeladene Videos abrufen (optional nach Profil filtern)
- DELETE /videos?profile_id=X — nicht heruntergeladene Videos des Profils loeschen
- DELETE /videos?profile_id=X&exclude_ids= — Videos des Profils loeschen (ausser lokal gespeicherte)
- POST /videos/{id}/download — Download triggern
- GET /videos/{id}/stream — Video streamen
- GET /videos/{id}/file — Video-Datei zum Download ausliefern
- DELETE /videos/{id}/file — Server-Datei loeschen (nach lokalem Download)
- WS /ws — WebSocket, benachrichtigt Clients bei neuen Videos
## Services
- VideoService — Videos speichern, abrufen, Status verwalten
- DownloadService — yt-dlp aufrufen, Video herunterladen
@@ -31,7 +33,7 @@
## Services
- LocalStorageService — Videos lokal speichern, pruefen, loeschen
## API
- ServerApi — kommuniziert mit FastAPI (GET /profiles, GET /videos, POST /download, GET /stream, GET /videos/{id}/file)
- ServerApi — kommuniziert mit FastAPI (GET /profiles, GET /videos, POST /download, GET /stream, GET /videos/{id}/file, DELETE /videos/{id}/file, DELETE /videos)
## Model
- Video — id, title, youtuber, thumbnailUrl, youtubeUrl, isDownloaded, localFilePath, profileIds
- Profile — id, name

View File

@@ -1,4 +1,4 @@
from fastapi import FastAPI
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from database import SessionLocal, create_tables
@@ -17,6 +17,32 @@ app.add_middleware(
app.include_router(videos_router)
app.include_router(profiles_router)
# --- WebSocket ---
connected_clients: set[WebSocket] = set()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
connected_clients.add(websocket)
try:
while True:
await websocket.receive_text()
except WebSocketDisconnect:
connected_clients.discard(websocket)
async def notify_clients(profile_ids: list[int]):
message = ",".join(str(pid) for pid in profile_ids)
for client in list(connected_clients):
try:
await client.send_text(message)
except Exception:
connected_clients.discard(client)
# --- Startup ---
@app.on_event("startup")
def startup():

View File

@@ -1,5 +1,5 @@
fastapi
uvicorn
uvicorn[standard]
yt-dlp
sqlalchemy
aiosqlite

View File

@@ -1,29 +1,39 @@
import asyncio
import threading
from pathlib import Path
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query
from fastapi.responses import FileResponse, StreamingResponse
from sqlalchemy.orm import Session
from database import get_db
from schemas import ProfileResponse, VideoCreate, VideoResponse
from schemas import CleanupRequest, ProfileResponse, VideoCreate, VideoResponse
from services import video_service
from services.download_service import download_video
from services.stream_service import stream_video_live
from services.video_service import update_file_path
router = APIRouter(prefix="/videos", tags=["videos"])
@router.post("", response_model=list[VideoResponse])
def create_videos(videos_data: list[VideoCreate], db: Session = Depends(get_db)):
async def create_videos(videos_data: list[VideoCreate], db: Session = Depends(get_db)):
created_ids = []
profile_ids = set()
for video_data in reversed(videos_data):
video_id_match = video_data.youtube_url.split("v=")[-1].split("&")[0]
video_service.delete_by_youtube_id(db, video_id_match)
video = video_service.create_video(db, video_data)
created_ids.append(video.id)
if video_data.profile_id:
profile_ids.add(video_data.profile_id)
videos = [video_service.get_video(db, vid) for vid in created_ids]
if profile_ids:
from main import notify_clients
await notify_clients(list(profile_ids))
return [VideoResponse.from_model(v) for v in videos if v]
@@ -39,9 +49,9 @@ def get_downloaded_videos(profile_id: Optional[int] = Query(None), db: Session =
return [VideoResponse.from_model(v) for v in videos]
@router.delete("")
def delete_not_downloaded(profile_id: int = Query(...), db: Session = Depends(get_db)):
count = video_service.delete_not_downloaded(db, profile_id)
@router.post("/cleanup")
def cleanup_videos(request: CleanupRequest, db: Session = Depends(get_db)):
count = video_service.delete_not_downloaded(db, request.profile_id, request.exclude_ids or None)
return {"deleted": count}
@@ -65,21 +75,23 @@ def stream_video(video_id: int, db: Session = Depends(get_db)):
raise HTTPException(status_code=404, detail="Video nicht gefunden")
if not video.file_path:
return StreamingResponse(
stream_video_live(video.youtube_url),
media_type="video/mp4",
)
def stream_and_save():
output_path = f"/videos/{video_id}.mp4"
yield from stream_video_live(video_id, video.youtube_url)
if Path(output_path).exists():
sdb = __import__("database").SessionLocal()
try:
update_file_path(sdb, video_id, output_path)
finally:
sdb.close()
return StreamingResponse(stream_and_save(), media_type="video/mp4")
path = Path(video.file_path)
if not path.exists():
raise HTTPException(status_code=404, detail="Videodatei nicht gefunden")
def iter_file():
with open(path, "rb") as f:
while chunk := f.read(1024 * 1024):
yield chunk
return StreamingResponse(iter_file(), media_type="video/mp4")
return FileResponse(path, media_type="video/mp4")
@router.get("/{video_id}/file")
@@ -97,6 +109,19 @@ def download_file(video_id: int, db: Session = Depends(get_db)):
return FileResponse(path, media_type="video/mp4", filename=f"{video.title}.mp4")
@router.delete("/{video_id}/file")
def delete_server_file(video_id: int, db: Session = Depends(get_db)):
video = video_service.get_video(db, video_id)
if not video:
raise HTTPException(status_code=404, detail="Video nicht gefunden")
if video.file_path:
path = Path(video.file_path)
if path.exists():
path.unlink()
video_service.update_file_path(db, video_id, None)
return {"status": "deleted"}
profiles_router = APIRouter(prefix="/profiles", tags=["profiles"])

View File

@@ -34,6 +34,11 @@ class VideoResponse(BaseModel):
)
class CleanupRequest(BaseModel):
profile_id: int
exclude_ids: list[int] = []
class ProfileResponse(BaseModel):
id: int
name: str

View File

@@ -12,7 +12,7 @@ def download_video(video_id: int, youtube_url: str):
subprocess.run(
[
"yt-dlp",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"-f", "bestvideo[ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]",
"-o", output_path,
"--merge-output-format", "mp4",
youtube_url,

View File

@@ -1,32 +1,53 @@
import subprocess
import time
from pathlib import Path
VIDEOS_DIR = "/videos"
def stream_video_live(youtube_url: str):
result = subprocess.run(
def stream_video_live(video_id: int, youtube_url: str):
output_path = f"{VIDEOS_DIR}/{video_id}.mp4"
path = Path(output_path)
process = subprocess.Popen(
[
"yt-dlp",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best",
"-g", youtube_url,
"-f", "best[ext=mp4][vcodec^=avc]/best[ext=mp4]",
"-o", output_path,
youtube_url,
],
capture_output=True, text=True, check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
urls = result.stdout.strip().split("\n")
cmd = ["ffmpeg"]
for url in urls:
cmd.extend(["-i", url])
cmd.extend(["-c", "copy", "-movflags", "frag_keyframe+empty_moov", "-f", "mp4", "pipe:1"])
# Warte bis Datei existiert und mindestens 1MB hat
while process.poll() is None:
if path.exists() and path.stat().st_size >= 1024 * 1024:
break
time.sleep(0.5)
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
try:
while True:
chunk = process.stdout.read(1024 * 1024)
if not chunk:
break
yield chunk
if not path.exists():
process.wait()
except GeneratorExit:
process.kill()
finally:
if process.poll() is None:
process.kill()
return
# Streame aus der wachsenden Datei
pos = 0
stall_count = 0
with open(output_path, "rb") as f:
while True:
chunk = f.read(1024 * 1024)
if chunk:
pos += len(chunk)
stall_count = 0
yield chunk
else:
if process.poll() is not None:
# Download fertig — restliche Bytes lesen
remaining = f.read()
if remaining:
yield remaining
break
stall_count += 1
if stall_count > 60: # 30 Sekunden ohne neue Daten
break
time.sleep(0.5)

View File

@@ -48,16 +48,20 @@ def update_file_path(db: Session, video_id: int, path: str):
db.commit()
def delete_not_downloaded(db: Session, profile_id: int) -> int:
videos = db.query(Video).filter(
Video.file_path.is_(None),
def delete_not_downloaded(db: Session, profile_id: int, exclude_ids: list[int] | None = None) -> int:
query = db.query(Video).filter(
Video.profiles.any(Profile.id == profile_id),
).all()
count = len(videos)
for video in videos:
db.delete(video)
)
if exclude_ids:
query = query.filter(Video.id.notin_(exclude_ids))
videos = query.all()
video_ids = [v.id for v in videos]
if not video_ids:
return 0
db.execute(video_profiles.delete().where(video_profiles.c.video_id.in_(video_ids)))
db.query(Video).filter(Video.id.in_(video_ids)).delete(synchronize_session=False)
db.commit()
return count
return len(video_ids)
def get_all_profiles(db: Session) -> list[Profile]:

View File

@@ -8,7 +8,8 @@
- Video herunterladen
# Server -> App
- Video streamen
- Videodatei senden
- Videodatei senden
- WebSocket: Benachrichtigung bei neuen Videos (Profile-IDs)
# App -> Server
- Profile abrufen
- Videos nach Profil abrufen

View File

@@ -12,4 +12,5 @@
- Klick auf Download lädt das Video herunter und wird lokal auf dem Client gespeichert
- Klick auf Icon zeigt verfügbare Profile
- Das ausgewählte Profil wird persistiert und bestimmt welche Videos angezeigt werden
- Klick auf Icon löscht alle nicht heruntergeladenen Videos vom aktuellen Profil
- Klick auf Icon löscht alle nicht heruntergeladenen Videos vom aktuellen Profil
- Bei neuen Videoeinträgen in der DB werden die Videos für das zugehörige Profil in der App aktualisiert