Location of the System

one-drive > finances > financial analysis

time to boot

it takes about five minutes following this process:

  • boot jekyll site - just cos i always do
  • boot new tmux session with
  • live server
  • watch script
  • ipthon session
  • windows vm so you can edit excel files

How the system works

  • we need a browser window with http://127.0.0.1:8080/
  • we load up live-server .
  • an ipython session is useful
  • we load up ….
  • i have some nix shells with ….. inside
  • we are using jupytext

live-server

live-server is a simple development HTTP server with live reload capability. It is commonly used for web development to automatically refresh the browser whenever files in the project directory change. This tool is particularly useful for front-end developers who want to see changes in real-time without manually refreshing the browser.

Key Features:

• Live Reloading: Automatically reloads the page when files change.
• Zero Configuration: Easy to set up and use with minimal configuration.
• Lightweight: Designed to be fast and efficient.

You can install it using npm.

analysis.py

The meat and potatoes occurs in this file. when we change it our python watcher script syncs it with ipynb and html. so lets look at the current structure of this file at the time i write:

  • imports
  • loads in lists:
  • things saving for; way i see it; questions; post january; financial buffers and my principles
  • functions
Function Notes
f_build_html_table  
f_build_html_from_dict  
f_build_sumstat_html  
f_filter_dataframe_by_string  
f_load_and_clean_expenses  
f_clean_amount_dataframe  
f_get_clean_range  
plot_data  
worksheet_names  
openpyxl_named_ranges_dict  
read_range_with_openpyxl  
extract_defined_names_from_dict  
list_to_html_bullets  
  • a whole series of jupyter cells that contain the anaylsis
    • in a way this is very bespoke
    • i populate variables with
      • worksheet names
      • excel named ranges
      • file paths
    • and probably most importantly
      • dataframes with cashflow information
        • date, amount,
Data Source Notes
Projected cashflows  
Expenses Table  

Output

The final outut is as follows:

  • Expenses Spreadsheet
  • balance charts
  • cashflows
Section Details
Expenses Spreadsheet  
Balance Charts  
Cashflows made up of 3 main sections: Cashflow tables; Summary Statistics; Questions to consider

lets consider the 3 cashflow subsections in more detail:

sub section details
cashflow tables 2 tables: regular cashflows and one off cashflows
summary statistics 9 sub sub sections: hmrc payments; salary payment; family aggregates; summary; to save for; buffers and principles; the way i see it; post january; monthly aggregates
questions to consider  

lets consider the 9 cashflow subsections in more detail:

Topic Notes
HMRC payments you have £00 per month until mid february
Salary payment 4.2k going into bank account monthly
Family aggregates aggegating monthly amounts to mum den and mary
Summary this sums up the different sub categories of expenses. quite interesting actually
To save for  
Buffers and principles  
The way I see it  
Post January  
Monthly aggregates  

jupytext

what is it

Jupytext is a Python package that allows you to convert Jupyter Notebooks to and from various text-based formats, such as Markdown, R Markdown, Python, and more. It enables version control of notebooks by storing them as plain text, making it easier to track changes using tools like Git. Jupytext can be used as a command-line tool or as a Jupyter Notebook extension, allowing seamless synchronization between the notebook and its text representation. This is particularly useful for collaborative projects and when integrating notebooks into software development workflows.

why use it

I believe i wanted to be able to modify code in neovim rather than a clunky jupyter webrowser interface

shell.nix 

{ pkgs ? import <nixpkgs> {} }:

let
  pythonEnv = pkgs.python3.withPackages (ps: with ps; [
#    pip

livereload
matplotlib
    tabulate
    numpy
    notebook
    pandas
    jupyter
    jupytext
    jupyterlab-git
    seaborn
    plotly
    openpyxl
    ipython
  ]);
in
pkgs.mkShell {
  buildInputs = [ pythonEnv
    pkgs.nodejs
    pkgs.nodePackages.live-server
  ];
}

notation.py, functions.py, and takeaways.py

what are these files?

watch.py

  • watch_directory is a function that monitors current directory for file changes.
  • it has 5 second intervals
  • takes snapshots to see if files have been moved modified or removed
  • changes are detailed in the console via print
  • Skips notation.py, functions.py, and takeaways.py.
  • Runs an automated pipeline: Sync .py ↔ .ipynb using jupytext. Execute the notebook in place (errors allowed). Export HTML versions (full and “clean” without inputs). Wait 8 seconds. Commit and push all changes to GitHub. Update baseline
import os
import subprocess
import time

def watch_directory(path='.'):                                                 
    before = {os.path.join(dp, f): os.path.getmtime(os.path.join(dp, f))       
              for dp, dn, filenames in os.walk(path) for f in filenames}       
    while True:                                                                
        time.sleep(5)                                                          
        after = {os.path.join(dp, f): os.path.getmtime(os.path.join(dp, f)) for dp, dn, filenames in os.walk(path) for f in filenames}    
        added = [f for f in after if f not in before]
        removed = [f for f in before if f not in after]
        modified = [f for f in after if f in before and before[f] != after[f]]

        if added:
            print("Added: ", ", ".join(added))
        if removed:
            print("Removed: ", ", ".join(removed))
        if modified:
            print("Modified: ", ", ".join(modified))

            if is_python_file(modified[0]) and modified[0] != 'notation.py' and modified[0] != 'functions.py'and modified[0] != 'takeaways.py':
                print ("this was a python file")
                run_bash_command("jupytext --sync "+str(modified[0]))  
                print("jupyter nbconvert --to notebook --inplace --execute --allow-errors "+remove_file_extension(modified[0])+".ipynb")
                run_bash_command("jupyter nbconvert --to notebook --inplace --execute --allow-errors "+remove_file_extension(modified[0])+".ipynb")
                print("jupyter nbconvert --to html " + remove_file_extension(modified[0])+".ipynb" + " --output "+ remove_file_extension(get_filename(modified[0]))+ ".html")
                run_bash_command("jupyter nbconvert --to html " + remove_file_extension(modified[0])+".ipynb" + " --output "+ remove_file_extension(get_filename(modified[0]))+ ".html")
                run_bash_command("jupyter nbconvert --to html --no-input --no-prompt " +remove_file_extension(modified[0])+".ipynb" + " --output "+ remove_file_extension(get_filename(modified[0])) + "-clean.html")
                print("done  -  in 8 seconds will push to github and take snapshots of timestamps")
                time.sleep(8)
                run_bash_command("git add .")
                run_bash_command("git commit -m '.'")
                run_bash_command("git push")
                after = {f: os.path.getmtime(f) for f in os.listdir(path)}
                added = [f for f in after if f not in before]
                removed = [f for f in before if f not in after]
                modified = [f for f in after if f in before and before[f] != after[f]]
        before = after
        print("waiting...")


def run_bash_command(command):
    result = subprocess.run(command, shell=True, capture_output=False, text=True)

def remove_file_extension(filename):
    return '.'.join(filename.split('.')[:-1]) if '.' in filename else filename

def is_python_file(filename):
    return filename.endswith('.py')


def get_filename(file_path):
    return os.path.basename(file_path)


watch_directory()


o

This site uses Just the Docs, a documentation theme for Jekyll.