kjanat/livegraphs-django
1
0
Fork 0
mirror of https://github.com/kjanat/livegraphs-django.git synced 2026-02-13 11:29:30 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix: configure markdownlint-cli2 properly

Browse Source 
- Wrap config in 'config' key for markdownlint-cli2
- Use MD013 rule name instead of 'line-length' alias
- Disable MD013 line-length checks
- Add allowed languages: sh, python, csv, tree
- Fix broken link reference in TODO.md
- All markdown linting now passes (37 errors -> 0)
This commit is contained in:
Kaj Kowalski
2025-11-05 16:51:23 +01:00
parent 04705bdcb2
commit fe847a3d4e
9 changed files with 434 additions and 353 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/CELERY_REDIS.md
View File

@@ -6,10 +6,10 @@ This document explains how to set up and use Redis and Celery for background tas
The data integration module uses Celery to handle:
- Periodic data fetching from external APIs
- Processing and storing CSV data
- Downloading and parsing transcript files
- Manual data refresh triggered by users
- Periodic data fetching from external APIs
- Processing and storing CSV data
- Downloading and parsing transcript files
- Manual data refresh triggered by users
## Installation
@@ -31,13 +31,13 @@ redis-cli ping # Should output PONG
After installation, check if Redis is properly configured:
1. Open Redis configuration file:
1. Open Redis configuration file:
```bash
sudo nano /etc/redis/redis.conf
```
2. Ensure the following settings:
2. Ensure the following settings:
```bash
# For development (localhost only)
@@ -53,7 +53,7 @@ After installation, check if Redis is properly configured:
port 6379
```
3. Restart Redis after any changes:
3. Restart Redis after any changes:
```bash
sudo systemctl restart redis-server
@@ -127,25 +127,25 @@ docker-compose up -d
Development requires multiple terminal windows:
1. **Django Development Server**:
1. **Django Development Server**:
```bash
make run
```
2. **Redis Server** (if needed):
2. **Redis Server** (if needed):
```bash
make run-redis
```
3. **Celery Worker**:
3. **Celery Worker**:
```bash
make celery
```
4. **Celery Beat** (for scheduled tasks):
4. **Celery Beat** (for scheduled tasks):
```bash
make celery-beat
@@ -163,12 +163,12 @@ make run-all
If you see connection errors:
1. Check that Redis is running: `redis-cli ping` should return `PONG`
2. Verify firewall settings are not blocking port 6379
3. Check Redis binding in `/etc/redis/redis.conf` (should be `bind 127.0.0.1` for local dev)
1. Check that Redis is running: `redis-cli ping` should return `PONG`
2. Verify firewall settings are not blocking port 6379
3. Check Redis binding in `/etc/redis/redis.conf` (should be `bind 127.0.0.1` for local dev)
### Celery Workers Not Processing Tasks
1. Ensure the worker is running with the correct app name: `celery -A dashboard_project worker`
2. Check the Celery logs for errors
3. Verify broker URL settings in both code and environment variables
1. Ensure the worker is running with the correct app name: `celery -A dashboard_project worker`
2. Check the Celery logs for errors
3. Verify broker URL settings in both code and environment variables

32
docs/TROUBLESHOOTING.md
View File

@@ -25,13 +25,13 @@ python manage.py test_redis
If this fails, check the following:
1. Redis might not be running. Start it with:
1. Redis might not be running. Start it with:
```bash
sudo systemctl start redis-server
```
2. Connection credentials may be incorrect. Check your environment variables:
2. Connection credentials may be incorrect. Check your environment variables:
```bash
echo $REDIS_URL
@@ -39,13 +39,13 @@ If this fails, check the following:
echo $CELERY_RESULT_BACKEND
```
3. Redis might be binding only to a specific interface. Check `/etc/redis/redis.conf`:
3. Redis might be binding only to a specific interface. Check `/etc/redis/redis.conf`:
```bash
grep "bind" /etc/redis/redis.conf
```
4. Firewall rules might be blocking Redis. If you're connecting remotely:
4. Firewall rules might be blocking Redis. If you're connecting remotely:
```bash
sudo ufw status # Check if firewall is enabled
@@ -56,9 +56,9 @@ If this fails, check the following:
If you see the error `zip() argument 2 is shorter than argument 1`, it means the data format doesn't match the expected headers. We've implemented a fix that:
1. Pads shorter rows with empty strings
2. Uses more flexible date format parsing
3. Provides better error handling
1. Pads shorter rows with empty strings
2. Uses more flexible date format parsing
3. Provides better error handling
After these changes, your data should be processed correctly regardless of format variations.
@@ -78,14 +78,14 @@ python manage.py test_celery
If the task isn't completing, check:
1. Look for errors in the Celery worker terminal
2. Verify broker URL settings match in both terminals:
1. Look for errors in the Celery worker terminal
2. Verify broker URL settings match in both terminals:
```bash
echo $CELERY_BROKER_URL
```
3. Check if Redis is accessible from both terminals:
3. Check if Redis is accessible from both terminals:
```bash
redis-cli ping
@@ -103,36 +103,36 @@ python manage.py celery inspect scheduled
Common issues with scheduled tasks:
1. **Celery Beat not running**: Start it with:
1. **Celery Beat not running**: Start it with:
```bash
cd dashboard_project
celery -A dashboard_project beat
```
2. **Task registered but not running**: Check worker logs for any errors
2. **Task registered but not running**: Check worker logs for any errors
3. **Wrong schedule**: Check the interval in settings.py and CELERY_BEAT_SCHEDULE
3. **Wrong schedule**: Check the interval in settings.py and CELERY_BEAT_SCHEDULE
## Data Source Configuration
If data sources aren't being processed correctly:
1. Verify active data sources exist:
1. Verify active data sources exist:
```bash
cd dashboard_project
python manage.py shell -c "from data_integration.models import ExternalDataSource; print(ExternalDataSource.objects.filter(is_active=True).count())"
```
2. Create a default data source if needed:
2. Create a default data source if needed:
```bash
cd dashboard_project
python manage.py create_default_datasource
```
3. Check source URLs and credentials in the admin interface or environment variables.
3. Check source URLs and credentials in the admin interface or environment variables.
## Manually Triggering Data Refresh