SDAP-484 - Cloud-optimized GeoTIFF support - #307

Draft

RKuttruff wants to merge 139 commits into

apache:developfrom

RKuttruff:SDAP-484-CoG

Commits on Jun 29, 2023

Separated NTS backends
rileykk
committed
6b1b7cc

Commits on Jul 5, 2023

n/a
rileykk
committed
4f3f611

Commits on Jul 6, 2023

More nts backend stuff
rileykk
committed
e32d5ad

Commits on Jul 10, 2023

Working(?) np backend
rileykk
committed
ccc0de4
Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
743fb1d
Working(?) np backend
rileykk
committed
b77aa11
gitignore ini
rileykk
committed
4ccec2e
ASF headers
rileykk
committed
736a44e

Commits on Jul 11, 2023

First functioning test of 2 simultaneous backends
rileykk
committed
70bdab1

Commits on Jul 12, 2023

Removed accidentally committed ini files
rileykk
committed
f3981cd
Working zarr backend ds list
rileykk
committed
26f6220
Capture and handle NTS requests routed to backend that doesn't (yet) support them
rileykk
committed
91de6ef
analysis setup fails to find VERSION.txt when building locally
rileykk
committed
df23919
Implemented more NTS functions in zarr backend
rileykk
committed
07404f0
Added misc backend time metrics record field in NCSH
rileykk
committed
72888aa

Commits on Jul 13, 2023

fixes
rileykk
committed
1c4a0e4

Commits on Jul 17, 2023

Dynamic dataset management
rileykk
committed
0a7cd7f

Commits on Jul 18, 2023

Dynamic dataset management
rileykk
committed
c8e7dbb

Commits on Jul 20, 2023

Dataset management
rileykk
committed
e78f7ad

Commits on Jul 27, 2023

Timeseriesspark support
rileykk
committed
a84d77e

Commits on Jul 31, 2023

Update backend dict on dataset mgmt query
rileykk
committed
53190e2
Fixes and improvements
rileykk
committed
2e7a0dc
Adapted matchup to work with zarr backends
rileykk
committed
0869375
Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
c156826

Commits on Aug 1, 2023

Zarr support
rileykk
committed
1eb680b

Commits on Aug 2, 2023

DDAS adjustments
rileykk
committed
0aef0f1

Commits on Aug 3, 2023

find_tile_by_polygon_and_most_recent_day_of_year impl
rileykk
committed
42b912e

Commits on Aug 8, 2023

Don't sel by time if neither max nor min time are given
rileykk
committed
1559fba

Commits on Aug 15, 2023

Fix not calling partial when needed
rileykk
committed
2bb52af

Commits on Aug 18, 2023

Pinned s3fs and fsspec versions
rileykk
committed
f9dc2ae
Fixed some dependencies to ensure image builds properly + s3fs works
rileykk
committed
a6f602d

Commits on Aug 21, 2023

Config override for backends
rileykk
committed
1a451eb
Deps update
rileykk
committed
6f8f7b1
Merge remote-tracking branch 'RKuttruff/master' into SDAP-472-gridded-zarr
rileykk
committed
5baf9ec

Commits on Aug 22, 2023

Commits on Aug 23, 2023

Commits on Aug 31, 2023

Add metadata from Zarr collection to /list
rileykk
committed
483ad9f

Commits on Sep 1, 2023

CoG backend start
rileykk
committed
e1dec65

Commits on Sep 5, 2023

Start of CoG work
rileykk
committed
bb2fa00
rioxarray dep
rileykk
committed
d57a531

Commits on Sep 6, 2023

CoG URL work and some Solr queries
rileykk
committed
3ad5530
Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
4b24ec3
Merge branch 'apache:master' into SDAP-484-CoG
RKuttruff
authored
87f7bc0
Merge remote-tracking branch 'RKuttruff/SDAP-484-CoG' into SDAP-484-CoG
rileykk
committed
6712c82

Commits on Sep 7, 2023

More functions implemented
rileykk
committed
aee843b
More functions implemented
rileykk
committed
c7b938d
Merge remote-tracking branch 'RKuttruff/SDAP-484-CoG' into SDAP-484-CoG
rileykk
committed
9d2605b
Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
6077ac2
Merge branch 'apache:master' into SDAP-484-CoG
RKuttruff
authored
a43db23

Commits on Sep 11, 2023

More CoG stuff
rileykk
committed
d51cf63

Commits on Sep 14, 2023

Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
0d3c0fc
Merge branch 'apache:master' into SDAP-484-CoG
RKuttruff
authored
f80496c
Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
8d51337
Merge branch 'apache:master' into SDAP-484-CoG
RKuttruff
authored
d26f4a2
Zarr: Probe lat order and flip if necessary
rileykk
committed
f5750c3
Zarr: Probe lat order and flip if necessary
rileykk
committed
b97677e

Commits on Sep 18, 2023

Fixes for subsetting
rileykk
committed
4cdb485

Commits on Sep 20, 2023

Warnings for geo subsetting
rileykk
committed
1ee25c2
Strip quotes from variable names
rileykk
committed
7fc260a
Strip quotes from variable names
rileykk
committed
62564f7

Commits on Sep 21, 2023

Fixed find_tile_by_id not routing to correct backend
rileykk
committed
b5a223b
Ensure geotiffs are sorted by dt
rileykk
committed
06504e0

Commits on Sep 25, 2023

removed resultSizeLimit param from matchup
stephenykp
committed
b5df944
Add # of primaries/avergae secondaries to job output
stephenykp
committed
5e0fbb2
rename to executionId
stephenykp
committed
fbad6b7
update changelog
stephenykp
committed
e0a5999

Commits on Sep 29, 2023

add totalSecondaryMatched field to /job output
stephenykp
committed
8942afc
num unique secondaries addition
stephenykp
committed
dd73036

Commits on Oct 3, 2023

Reenabled TIFF geo subsetting
rileykk
committed
a7beb85
Ensure latitudes are in ascending order
rileykk
committed
741d4b6

Commits on Oct 9, 2023

Catch and report algs that are unsupported by ds backend
rileykk
committed
5003420

Commits on Oct 10, 2023

Catch and report algs that are unsupported by ds backend
rileykk
committed
15fce2e

Commits on Oct 12, 2023

Code for opening tiffs in S3
rileykk
committed
d0742ad

Commits on Oct 13, 2023

updated docs to use correct sea_water_temperature param name
stephenykp
committed
db68d4f

Commits on Oct 17, 2023

Merge remote-tracking branch 'origin/master' into SDAP-484-CoG
rileykk
committed
5e77bae

Commits on Oct 26, 2023

Merge branch 'apache:master' into SDAP-484-CoG
RKuttruff
authored
29f556f

Commits on Nov 1, 2023

Merge remote-tracking branch 'origin' into SDAP-493
stephenykp
committed
7e11a4c
bugfix
stephenykp
committed
a8be9b8

Commits on Nov 6, 2023

fix division by zero bug
stephenykp
committed
62de867

Commits on Nov 8, 2023

add params to dataset management handler classes
rileykk
committed
972f3dd
Merge remote-tracking branch 'origin/master' into SDAP-472-gridded-zarr
rileykk
committed
9f0a107

Commits on Nov 16, 2023

add page number to default filename for matchup output
stephenykp
committed
831ca37
pagination improvements
stephenykp
committed
4ab2f9b
removed debugging line
stephenykp
committed
3677c11
changelog
stephenykp
committed
86f1348

Commits on Nov 28, 2023

Update helm cassandra dependency (#289)

authored and
stephenykp
committed
1e8cc4e

Commits on Jan 5, 2024

stac catalog
stephenykp
committed
32ca3d7

Commits on Jan 6, 2024

Updated openapi spec
stephenykp
committed
3563ae9
move stac endpoints to matchup tag in openapi spec
stephenykp
committed
0691d87

Commits on Jan 11, 2024

SDAP-507 - Changes to remove geos sub-dependency
rileykk
committed
e02fc78
SDAP-507 - Changelog
rileykk
committed
51231ca
SDAP-507 - Changes to remove geos sub-dependency
rileykk
committed
5c75573
SDAP-507 - Changelog
rileykk
committed
7f717c0
Merge remote-tracking branch 'origin/SDAP-507' into SDAP-507
rileykk
committed
2b6efa6

Commits on Jan 19, 2024

delete instead of comment out
rileykk
committed
9779f40
Merge branch 'SDAP-500' into SDAP-499
stephenykp
committed
3e700b8
Revert "Update helm cassandra dependency (#289)"
stephenykp
committed
9378760
Merge branch 'SDAP-499' into SDAP-506
stephenykp
committed
092c87b
Merge branch 'release/1.2.0' into SDAP-506
stephenykp
committed
e6730eb
Merge remote-tracking branch 'origin/SDAP-506' into SDAP-472-gridded-zarr
rileykk
committed
3aafb7e
deleted disabled endpoint files
rileykk
committed
5303146
Merge branch 'SDAP-507' into SDAP-472-gridded-zarr
rileykk
committed
d6a75e3
Merge branch 'develop' into SDAP-493
stephenykp
committed
2ed29fd
Merge branch 'SDAP-493' into SDAP-500
stephenykp
committed
2a340dc
Merge branch 'SDAP-500' into SDAP-499
stephenykp
committed
718067c
Merge branch 'SDAP-499' into SDAP-506
stephenykp
committed
681ba5a
Merge branch 'release/1.2.0' into SDAP-472-gridded-zarr
rileykk
committed
935000b

Commits on Jan 25, 2024

Commits on Jan 29, 2024

Merge remote-tracking branch 'origin/SDAP-506' into SDAP-472-gridded-zarr
rileykk
committed
6bd5f0e

Commits on Feb 1, 2024

Update .asf.yaml (#293)

RKuttruff
and
rileykk
authored
639d7d7
Merge branch 'apache:master' into SDAP-472-gridded-zarr
RKuttruff
authored
5937db1
Merge remote-tracking branch 'origin/develop' into SDAP-472-gridded-zarr
rileykk
committed
3050ef3
Moved changelog entries
rileykk
committed
3875f2d
SDAP-472 changelog entries
rileykk
committed
d989c66

Commits on Feb 6, 2024

Merge branch 'SDAP-472-gridded-zarr' into SDAP-484-CoG
rileykk
committed
2c22be6

Commits on Feb 8, 2024

Merge branch 'develop' into SDAP-472-gridded-zarr
rileykk
committed
f766df1

Commits on Feb 12, 2024

Merge branch 'SDAP-472-gridded-zarr' into SDAP-484-CoG
rileykk
committed
3285c1e
pyproj requirement
rileykk
committed
ec2ed11

Commits on Feb 13, 2024

CoG: some minor fixes for docker build &c
rileykk
committed
1392983

Commits on Mar 6, 2024

Merge branch 'develop' into SDAP-472-gridded-zarr
RKuttruff
authored
1ce1732

Commits on Apr 1, 2024

Merge branch 'SDAP-472-gridded-zarr' into SDAP-484-CoG
rileykk
committed
42530a2
Merge remote-tracking branch 'origin/develop' into SDAP-484-CoG
rileykk
committed
c1c4c49
Dependencies update to poetry
rileykk
committed
168ae73

Commits on Apr 2, 2024

Handling of GeoTIFFs with non-float dtypes
rileykk
committed
cefa8ab

Commits on Apr 16, 2024

Improved opening of GeoTIFFs to avoid mask & scale issues with dtype
rileykk
committed
2800569

Commits on May 6, 2024

Merge branch 'develop' into SDAP-484-CoG
rileykk
committed
6b6bb75

Commits on Jun 10, 2024

Merge branch 'develop' into SDAP-484-CoG
rileykk
committed
d1fb37c
Poetry re-lock
rileykk
committed
18bbfea
Fix bad merge
rileykk
committed
9a11bad

Commits on Jun 25, 2024

Merge branch 'apache:develop' into SDAP-484-CoG
RKuttruff
authored
d44e530

Commits on Jul 10, 2024

Merge remote-tracking branch 'origin/develop' into SDAP-484-CoG
rileykk
committed
f41ebe6
poetry lock
rileykk
committed
c7e6e82

Commits on Oct 8, 2024

Merge remote-tracking branch 'origin/develop' into SDAP-484-CoG
rileykk
committed
606e13f
Poetry re-lock
rileykk
committed
f2e5cf1
Update backend for new NTS method
rileykk
committed
0cb9c83

Commits on Oct 14, 2024

Merge branch 'apache:develop' into SDAP-484-CoG
RKuttruff
authored
1e746f8