Code

(draft) En analysis#35

Code

Draft

justaddcoffee wants to merge 78 commits intomainmonarch-initiative/pheval.llm:mainfrom short_lettermonarch-initiative/pheval.llm:short_letterCopy head branch name to clipboard

+1,926,429-91

Commits

Commits on Jul 1, 2024

• Make outdir if it doesn't exist

  

  Justin Reese

  committed
  2d89b7b

  Copy full SHA for 2d89b7b

• Only English for this analysis

  

  Justin Reese

  committed
  afbfbd9

  Copy full SHA for afbfbd9

• Add input and output from July run of malco

  

  Justin Reese

  committed
  41be8e6

  Copy full SHA for 41be8e6

• Remove parallelization, add plot for top-k accuracy

  

  Justin Reese

  committed
  14ca7d1

  Copy full SHA for 14ca7d1

• Hard code path to results file

  

  Justin Reese

  committed
  9e5709d

  Copy full SHA for 9e5709d

Commits on Jul 2, 2024

• Add text mined results

  

  Justin Reese

  committed
  42bab8d

  Copy full SHA for 42bab8d

Commits on Jul 3, 2024

• Correct order of args when calling ontogpt

  

  Justin Reese

  committed
  c151b25

  Copy full SHA for c151b25

Commits on Jul 6, 2024

• Update ontogpt

  

  Justin Reese

  committed
  af2a2db

  Copy full SHA for af2a2db

• Add results for (all 5,269) phenopackets

  

  Justin Reese

  committed
  183e700

  Copy full SHA for 183e700

Commits on Jul 8, 2024

• Remove incomplete results

  

  Justin Reese

  committed
  54a7bbe

  Copy full SHA for 54a7bbe

• Update script to make fig 2

  

  Justin Reese

  committed
  58a5339

  Copy full SHA for 58a5339

• Notebook to calculate unique HPO terms and terms per case

  

  Justin Reese

  committed
  37e524b

  Copy full SHA for 37e524b

• NB to make fig 2

  

  Justin Reese

  committed
  ad2180b

  Copy full SHA for ad2180b

• Change y axis on fig

  

  Justin Reese

  committed
  8e5d3b8

  Copy full SHA for 8e5d3b8

Commits on Jul 10, 2024

• Update summary statistics nb

  

  Justin Reese

  committed
  f78eedc

  Copy full SHA for f78eedc

• Add code to calculate % at rank 1, 3, 10

  

  Justin Reese

  committed
  6415c20

  Copy full SHA for 6415c20

• Use R instead of python for plot

  

  Justin Reese

  committed
  ad54d66

  Copy full SHA for ad54d66

• Tweaks to R plot

  

  Justin Reese

  committed
  211f902

  Copy full SHA for 211f902

• Add JAMA color theme

  

  Justin Reese

  committed
  640a5d0

  Copy full SHA for 640a5d0

Commits on Jul 11, 2024

• Commit text_mined phenopackets and prompts

  

  Justin Reese

  committed
  5b87556

  Copy full SHA for 5b87556

Commits on Jul 15, 2024

• Remove text-mined stuff

  

  Justin Reese

  committed
  55b3a84

  Copy full SHA for 55b3a84

• Change plot to only show phenopacket (not text-mined) cases

  

  Justin Reese

  committed
  ce3efcd

  Copy full SHA for ce3efcd

• Tweaks to plot

  

  Justin Reese

  committed
  18c4cd8

  Copy full SHA for 18c4cd8

Commits on Jul 16, 2024

• Update fig 2

  

  Justin Reese

  committed
  678bd3f

  Copy full SHA for 678bd3f

Commits on Jul 26, 2024

• Update notebook

  

  Justin Reese

  committed
  7b64f8d

  Copy full SHA for 7b64f8d

Commits on Aug 5, 2024

• mini script looking at how many times in published output the grounding to mondo succeeded

  leokim-l

  committed
  739f346

  Copy full SHA for 739f346

Commits on Aug 29, 2024

• Add code to process exomiser results

  

  Justin Reese

  committed
  26b81e6

  Copy full SHA for 26b81e6

Commits on Aug 30, 2024

• Parallelize processing exomiser results to get time down to hours instead of days

  

  Justin Reese

  committed
  fcbc65e

  Copy full SHA for fcbc65e

Commits on Sep 17, 2024

• Preliminary Exomiser results (not apples to apples with 4o data, since we didn't use Mondo to find fuzzy matches

  

  Justin Reese

  committed
  8cf281b

  Copy full SHA for 8cf281b

• Add stuff to gitignore

  

  Justin Reese

  committed
  912d65c

  Copy full SHA for 912d65c

• Script to make hits at n for exomiser

  

  Justin Reese

  committed
  c6223e0

  Copy full SHA for c6223e0

• Add stuff to .gitignore

  

  Justin Reese

  committed
  52e2ed9

  Copy full SHA for 52e2ed9

Commits on Sep 19, 2024

• Update nb to correcting rank Exomiser predictions

  

  Justin Reese

  committed
  06a70fb

  Copy full SHA for 06a70fb

Commits on Sep 20, 2024

• Update notebook to finalize and analyze Exomiser results

  

  Justin Reese

  committed
  a3aefc0

  Copy full SHA for a3aefc0

• Merge branch 'short_letter' of https://github.com/monarch-initiative/malco into short_letter

  

  Justin Reese

  committed
  57b5626

  Copy full SHA for 57b5626

• Rename exomiser nb to be more descriptive

  

  Justin Reese

  committed
  0a094fe

  Copy full SHA for 0a094fe

Commits on Sep 22, 2024

• Working on grounding o1 results

  

  Justin Reese

  committed
  80ebfda

  Copy full SHA for 80ebfda

• Ground with Mondo

  

  Justin Reese

  committed
  75aee6c

  Copy full SHA for 75aee6c

• Add plots comparing the models

  

  Justin Reese

  committed
  022b9a3

  Copy full SHA for 022b9a3

Commits on Sep 23, 2024

• Fix bug writing out correct ID to output files

  

  Justin Reese

  committed
  9d7713d

  Copy full SHA for 9d7713d

Commits on Sep 24, 2024

• Update o1 analysis nb

  

  Justin Reese

  committed
  c922de0

  Copy full SHA for c922de0

• Add better grounding. Now o1 responses are about 98.4% grounded

  

  Justin Reese

  committed
  808dd1b

  Copy full SHA for 808dd1b

• Add better grounding. Now o1 responses are about 98.4% grounded

  

  Justin Reese

  committed
  1ede4f9

  Copy full SHA for 1ede4f9

Commits on Sep 25, 2024

• Refactor grounding, no inexact matching, because it grounds spuriously

  

  Justin Reese

  committed
  e2732f6

  Copy full SHA for e2732f6

Commits on Sep 26, 2024

• Updated nb, died during run

  

  Justin Reese

  committed
  d8e8b47

  Copy full SHA for d8e8b47

• Rerun analysis and plotting of o1 with exact match and split on ()

  

  Justin Reese

  committed
  fcf3969

  Copy full SHA for fcf3969

• Working on better grounding for o1

  

  Justin Reese

  committed
  e534b0b

  Copy full SHA for e534b0b

Commits on Sep 27, 2024

• Update nb and results using OAK then curategpt for grounding

  

  Justin Reese

  committed
  8498a33

  Copy full SHA for 8498a33

• Fix typo

  

  Justin Reese

  committed
  aa18420

  Copy full SHA for aa18420

• Fix typo

  

  Justin Reese

  committed
  bc06de8

  Copy full SHA for bc06de8

• Add comment about how to create mondo collection using curategpt

  

  Justin Reese

  committed
  43cb5cc

  Copy full SHA for 43cb5cc

Commits on Sep 30, 2024

• Add nb to (re)process gpt-4o

  

  Justin Reese

  committed
  60f1444

  Copy full SHA for 60f1444

Commits on Oct 1, 2024

• Notebook to make and analyze GPT-4o results (WIP)

  

  Justin Reese

  committed
  0bfdd4b

  Copy full SHA for 0bfdd4b

• Add gpt 4o analysis

  

  Justin Reese

  committed
  eb236c2

  Copy full SHA for eb236c2

• Add o4 results to plot

  

  Justin Reese

  committed
  861856e

  Copy full SHA for 861856e

• Comment about 4o version

  

  Justin Reese

  committed
  3ef9ee6

  Copy full SHA for 3ef9ee6

Commits on Oct 2, 2024

• added prompts which hopefully will be the final ground thruth version for short letter

  leokim-l

  committed
  60fd23a

  Copy full SHA for 60fd23a

Commits on Oct 10, 2024

• Writing nb to calculate stats for what diseases were found / not found by Exomiser and o1

  

  Justin Reese

  committed
  e95a15a

  Copy full SHA for e95a15a

Commits on Oct 15, 2024

• Add o1 mini analysis, rename plot_exomiser_o1_4turbo.ipynb -> plot_exomiser_o1_4o.ipynb for clarity

  

  Justin Reese

  committed
  62b5361

  Copy full SHA for 62b5361

Commits on Oct 16, 2024

• Add o1 mini notebook

  

  Justin Reese

  committed
  7f03bd8

  Copy full SHA for 7f03bd8

• Add WIP mcnemars analysis

  

  Justin Reese

  committed
  1bf35bf

  Copy full SHA for 1bf35bf

• Add o1 preview results

  

  Justin Reese

  committed
  d2f213f

  Copy full SHA for d2f213f

• Tweak legend

  

  Justin Reese

  committed
  187e121

  Copy full SHA for 187e121

• Tweak legend

  

  Justin Reese

  committed
  689cdbd

  Copy full SHA for 689cdbd

Commits on Oct 24, 2024

• Tweak figure

  

  Justin Reese

  committed
  aeb9f6a

  Copy full SHA for aeb9f6a

Commits on Oct 26, 2024

• Update plot_exomiser_o1MINI_o1PREVIEW_4o.ipynb

  jmcmurry

  authored
  7b06b4e

  Copy full SHA for 7b06b4e

• Merge pull request #53 from monarch-initiative/jmcmurry-patch-1

  Show description for dc43c19
  jmcmurry

  authored
  dc43c19

  Copy full SHA for dc43c19

Commits on Oct 29, 2024

• Remove new (unused) prompts dir

  

  Justin Reese

  committed
  73a4b5d

  Copy full SHA for 73a4b5d

• Tweak colors

  

  Justin Reese

  committed
  db3a395

  Copy full SHA for db3a395

• Tidy up notebook dir

  

  Justin Reese

  committed
  a9395b9

  Copy full SHA for a9395b9

Commits on Dec 10, 2024

• Update notebooks with various improvements

  

  Justin Reese

  committed
  2a1ff5d

  Copy full SHA for 2a1ff5d

• Add script to generate disease grounding summary

  

  Justin Reese

  committed
  7ae245c

  Copy full SHA for 7ae245c

Commits on Dec 12, 2024

• Add process_gpt_o1_VIGNETTE.ipynb (WIP()

  

  Justin Reese

  committed
  df1d7eb

  Copy full SHA for df1d7eb

• Checkpoint

  

  Justin Reese

  committed
  b565625

  Copy full SHA for b565625

Commits on Dec 13, 2024

• Fix regex

  

  Justin Reese

  committed
  aa7be84

  Copy full SHA for aa7be84

• Add tqdm progress bar

  

  Justin Reese

  committed
  cc858ec

  Copy full SHA for cc858ec

Commits on Dec 16, 2024

• Add gpt results

  

  Justin Reese

  committed
  a9fe0de

  Copy full SHA for a9fe0de

Commits on Jan 6, 2025

• Commit results files

  

  Justin Reese

  committed
  4433eac

  Copy full SHA for 4433eac