Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
do-me committed Sep 9, 2024
1 parent f1f9537 commit 8980ef9
Show file tree
Hide file tree
Showing 9,102 changed files with 30,493 additions and 1 deletion.
The diff you're trying to view is too large. We only load the first 3000 changed files.
22 changes: 22 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: ci
on:
push:
branches:
- main
permissions:
contents: write
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure Git Credentials
run: |
git config user.name github-actions[bot]
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
- uses: actions/setup-python@v5
with:
python-version: 3.x
- run: pip install "mkdocs-material[imaging]==9.5.34"
- run: pip install mkdocs-glightbox mkdocs-minify-plugin mkdocs-git-revision-date-localized-plugin mkdocs-git-committers-plugin-2
- run: mkdocs gh-deploy --force
165 changes: 165 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# MacOS stuff
.DS_Store

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
.pdm.toml
.pdm-python
.pdm-build/

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
20 changes: 20 additions & 0 deletions README copy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# KCEO Glossary

Test repo for KCEO glossary

## Mac build error
If error `ERROR - "cairosvg" Python module is installed, but it crashed with:` just use:

`export DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib` before. Or set it once, e.g. for zsh shell:

`echo 'export DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib' >> ~/.zshrc && source ~/.zshrc`

## Installation commands
See ci.yml.

## Contributions and Feedback
The KCEO Glossary is freely available for you to use. In addition, your feedback and ideas are welcome to improve the terms and create alternative versions.

If you have questions you can ping us or open an [Issue] (add link).

Feel free to also send us a pull request.
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
# KCEO Glossary

Currently work in progress.
## Contributions and Feedback
The KCEO Glossary is built for the EO community. Your feedback and ideas are fundamental for the further development of this glossary. PR's and contributions of any kind are highly welcomed.

### Contribution Guide
coming soon

## Local installation
See the github actions in .github/workflows/ci.yml.

## FAQ
### Mac build error
If error `ERROR - "cairosvg" Python module is installed, but it crashed with:` just use:

`export DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib` before. Or set it once, e.g. for zsh shell:

`echo 'export DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib' >> ~/.zshrc && source ~/.zshrc`

76 changes: 76 additions & 0 deletions add_cross_links.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
import os
import re

def extract_titles_from_md(file_path):
titles = []
with open(file_path, 'r') as file:
in_yaml = False
for line in file:
if line.strip() == '---':
in_yaml = not in_yaml
if in_yaml:
continue
if line.startswith('# ') and not line.startswith('##'):
# Remove "#" and any potential markdown link syntax
title = re.sub(r'\[([^\]]+)\]\(.*?\)', r'\1', line.lstrip('# ').strip())
titles.append(title)
return titles

def insert_hyperlinks(file_path, terms):
with open(file_path, 'r') as file:
content = file.readlines()

new_content = []
in_yaml = False
current_title = None

for line in content:
if line.strip() == '---':
in_yaml = not in_yaml
new_content.append(line)
continue
if in_yaml:
new_content.append(line)
continue
if line.startswith('# ') and not line.startswith('##'):
current_title = re.sub(r'\[([^\]]+)\]\(.*?\)', r'\1', line.lstrip('# ').strip())
new_content.append(line)
continue

for term in terms:
if term == current_title: # Skip self-references
continue

# Skip terms that already have a hyperlink
if re.search(r'\[' + re.escape(term) + r'\]\(.*?\)', line):
continue

# Avoid altering headings by ensuring replacements are only made in non-heading lines
pattern = re.compile(r'\b' + re.escape(term) + r'\b', re.IGNORECASE)
replacement = f'[{term}](../{" ".join(term.split()).lower()})'
line = pattern.sub(lambda match: replacement if not re.search(rf'\[{re.escape(match.group(0))}\]\(.*?\)', match.string) else match.group(0), line)

new_content.append(line)

with open(file_path, 'w') as file:
file.writelines(new_content)

def main(directory, exclude_files):
all_titles = []
md_files = [f for f in os.listdir(directory) if f.endswith('.md') and f not in exclude_files]

# Extract titles from all markdown files
for md_file in md_files:
file_path = os.path.join(directory, md_file)
titles = extract_titles_from_md(file_path)
all_titles.extend(titles)

# Insert hyperlinks based on extracted titles
for md_file in md_files:
file_path = os.path.join(directory, md_file)
insert_hyperlinks(file_path, all_titles)

if __name__ == "__main__":
directory = './docs' # Replace with your directory path
exclude_files = ['changelog.md', 'impressum.md', 'index.md', 'glossary.md', 'introduction.md', 'mermaid.md'] # Files to be excluded
main(directory, exclude_files)
50 changes: 50 additions & 0 deletions create_topology.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import re
from collections import defaultdict

# Helper function to parse the mermaid markdown and extract relationships
def parse_mermaid_relationships(mermaid_content):
# Adjusted regex pattern
pattern = re.compile(r'id_([\w-]+)\(\["<a href=\'\.\./([\w-]+)\'>([\w\s-]+)</a>"\]\) --> id_([\w-]+)\(\["<a href=\'\.\./([\w-]+)\'>([\w\s-]+)</a>"\]\)')
relationships = defaultdict(lambda: {'parents': set(), 'children': set(), 'link': None})

for match in pattern.finditer(mermaid_content):
parent_id, parent_link, parent_name, child_id, child_link, child_name = match.groups()

# Add child to parent's children list
relationships[parent_name]['children'].add((child_name, f"../{child_link}/"))
relationships[parent_name]['link'] = f"../{parent_link}/" # Add link for parent

# Add parent to child's parents list
relationships[child_name]['parents'].add((parent_name, f"../{parent_link}/"))
relationships[child_name]['link'] = f"../{child_link}/" # Add link for child

return relationships

# Read the mermaid.md file
with open('docs/mermaid.md', 'r') as file:
mermaid_content = file.read()

# Remove leading and trailing backticks and optional whitespace/newlines
mermaid_content = mermaid_content.strip('` \n')

# Parse relationships
relationships = parse_mermaid_relationships(mermaid_content)

# Write the glossary_topology.md file
with open('docs/glossary_topology.md', 'w') as file:
for term, links in sorted(relationships.items()):
file.write(f'## [{term}]({links["link"]})\n') # Make heading a hyperlink
if links['parents']:
file.write('**Parents**: ')
file.write(', '.join(f"[{parent[0]}]({parent[1]})" for parent in sorted(links['parents'])))
if links['children']:
file.write('<br>\n')
else:
file.write('\n')
if links['children']:
file.write('**Children**: ')
file.write(', '.join(f"[{child[0]}]({child[1]})" for child in sorted(links['children'])))
file.write('\n')
file.write('\n')

print('glossary_topology.md has been created/overwritten with the extracted relationships.')
Loading

0 comments on commit 8980ef9

Please sign in to comment.