Ported queries to be compatible with QLever by fenymufyd · Pull Request #661 · govdirectory/website

fenymufyd · 2026-07-15T13:17:26Z

Description

I've ported all .rq query files to be compatible with QLever. I used LLM (Copilot) to do most of the tedious work but I've tested some samples of the queries manually on WQDS and QLever and it works on both services. I think it also made organization-optional.rq faster on QLever, about 5-8 sec per item.

Issue Reference

this should fix #498

Checklist

Please ensure the following before submitting the PR:

I have tested the changes locally, and they work as expected.
The code follows the project's coding standards and conventions.
I have updated relevant documentation (if needed).
I have added unit tests or performed manual testing where necessary.

Abbe98

Thank you so much for this work! Here are a few first notes.

fenymufyd

reviewed all except countries.rq

Abbe98 · 2026-07-15T14:48:55Z

Thank you @fenymufyd, I have some time scheduled for Govdirectory on Friday and will try to get a in depth review done!

Added a comment to clarify the purpose of the OPTIONAL block.

Abbe98 · 2026-07-17T13:26:24Z

I'm rather happy now with this now! I want to merge #655 first and then rebase this. Not sure if I will be able to get it over the finish line before/during Wikimania but I hope so.

Abbe98 · 2026-09-02T19:39:08Z

EDIT: I refactored they query and it's now much faster. I haven't been able to get the following query through Qlever yet:

# expected_result_count: 429

PREFIX p: <http://www.wikidata.org/prop/>
PREFIX pq: <http://www.wikidata.org/prop/qualifier/>
PREFIX ps: <http://www.wikidata.org/prop/statement/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX schema: <http://schema.org/>
PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wdno: <http://www.wikidata.org/prop/novalue/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>

SELECT DISTINCT
  ?qid
  ?orgLabel
  ?orgDescription
  ?type
  ?typeLabel
  ?country
WHERE {
  BIND(wd:Q55 AS ?country)

  {
  # agencies with country set to the Netherlands
  VALUES ?type {
    wd:Q3143387 # ministries (12)
    wd:Q4500821 # appeals court (4)
    wd:Q190752  # supreme court (2)
    wd:Q2125157 # district court (11)
    wd:Q134390  # provinces (12)
    wd:Q702081 # water boards (21)
    wd:Q2039348 # municipalities (342)
    wd:Q26934845 # integral overseas territory (3)
  }
  ?org wdt:P31 ?type .

  ?org wdt:P17 ?country .
  } UNION {
    # special case without a unique type
    VALUES ?org {
      wd:Q1857094 # cabinet of the King (1)
    }

    BIND("cabinet of the King" AS ?typeLabelCabinet)
  }
  UNION {
    ?org wdt:P31 wd:Q947233 . # high council of the state (6)
    BIND("High Council of State" AS ?typeLabelCouncil)
  }
  UNION {
    ?org wdt:P31 wd:Q138643360 . # inspectorates (15)
    BIND("Inspectorate" AS ?typeLabelInspectorate)
  }

  MINUS { ?org wdt:P576 [] . MINUS { ?item rdf:type wdno:P576 } }
  MINUS { ?org wdt:P1366 [] }
  MINUS { ?org wdt:P582 [] .
          MINUS { ?org wdt:P582 ?ddate ;
                       wdt:P580 ?idate .
                  FILTER ( ?idate > ?ddate )
               }
        }
  MINUS { 
    ?org p:P31 ?inStatement .
    ?inStatement pq:P582 [] .
    MINUS { ?org p:P31 ?p.
            ?p ps:P31 wd:Q2039348; pq:P580 ?start .
            FILTER NOT EXISTS { ?p pq:P582 []}
    }             
  }
  MINUS { ?org p:P31 ?p.
            ?p ps:P31 wd:Q2039348; pq:P580 ?start .
            FILTER ( ?start > NOW() )
  }

  BIND(REPLACE(STR(?org), "http://www.wikidata.org/entity/", "") AS ?qid)

  OPTIONAL { ?org rdfs:label ?orgLabelEn FILTER(LANG(?orgLabelEn) = "en") }
  OPTIONAL { ?org rdfs:label ?orgLabelNl FILTER(LANG(?orgLabelNl) = "nl") }
  OPTIONAL { ?org rdfs:label ?orgLabelMul FILTER(LANG(?orgLabelMul) = "mul") }
  BIND(COALESCE(?orgLabelEn, ?orgLabelNl, ?orgLabelMul, STR(?org)) AS ?orgLabel)

  OPTIONAL { ?org schema:description ?orgDescriptionEn FILTER(LANG(?orgDescriptionEn) = "en") }
  OPTIONAL { ?org schema:description ?orgDescriptionNl FILTER(LANG(?orgDescriptionNl) = "nl") }
  OPTIONAL { ?org schema:description ?orgDescriptionMul FILTER(LANG(?orgDescriptionMul) = "mul") }
  BIND(COALESCE(?orgDescriptionEn, ?orgDescriptionNl, ?orgDescriptionMul) AS ?orgDescription)

  OPTIONAL { ?type rdfs:label ?typeLabelEn FILTER(LANG(?typeLabelEn) = "en") }
  OPTIONAL { ?type rdfs:label ?typeLabelNl FILTER(LANG(?typeLabelNl) = "nl") }
  OPTIONAL { ?type rdfs:label ?typeLabelMul FILTER(LANG(?typeLabelMul) = "mul") }
  BIND(COALESCE(?typeLabelCabinet, ?typeLabelCouncil, ?typeLabelInspectorate, ?typeLabelEn, ?typeLabelNl, ?typeLabelMul, STR(?type)) AS ?typeLabel)
}
ORDER BY ?type ?typeLabel ?orgLabel

fenymufyd · 2026-09-14T13:15:43Z

@Abbe98 this will fix the query so it can run on QLever. It seems we need to bind custom union type like Q1857094 to ?type first then bind that type label to custom ?typeLabel2 so the final bind coalesce works below it BIND(COALESCE(?typeLabel2, ?typeLabelEn, ?typeLabelNl, ?typeLabelMul, STR(?type)) AS ?typeLabel)

I also moved the wd:Q947233 # High Council of State (6) union to normal ?type above it because no value is being uncounted, which also saves a bit more time (around 200-500ms iirc)

# expected_result_count: 429

PREFIX p: <http://www.wikidata.org/prop/>
PREFIX pq: <http://www.wikidata.org/prop/qualifier/>
PREFIX ps: <http://www.wikidata.org/prop/statement/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX schema: <http://schema.org/>
PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wdno: <http://www.wikidata.org/prop/novalue/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>

SELECT DISTINCT
  ?qid
  ?orgLabel
  ?orgDescription
  ?type
  ?typeLabel
  ?country
WHERE {
  BIND(wd:Q55 AS ?country) # agencies with country set to the Netherlands

  {

    VALUES ?type { 
      wd:Q3143387 # ministries (12)
      wd:Q4500821 # appeals court (4)
      wd:Q190752  # supreme court (2)
      wd:Q2125157 # district court (11)
      wd:Q134390  # provinces (12)
      wd:Q702081 # water boards (21)
      wd:Q2039348 # municipalities (342)
      wd:Q26934845 # integral overseas territory (3)
      wd:Q947233 # High Council of State (6)
    }   
    
    ?org wdt:P31 ?type .
    
    ?org wdt:P17 ?country .
  } 
  UNION {
    VALUES ?org { wd:Q1857094 } # all instances of "Kabinet van de Koning" of netherlands (1)
    BIND(wd:Q1857094 AS ?type)  # bind custom type to ?type first
    BIND("cabinet of the King" AS ?typeLabel2)
  }
  UNION {
    ?org wdt:P31 wd:Q138643360 . # all instances of "Inspectorate" of netherlands, whether from mainland country (Q55) or the transcontinental kingdom (Q29999) (15)
    BIND(wd:Q138643360 AS ?type)
    BIND("Inspectorate" AS ?typeLabel2)
  }

  MINUS { ?org wdt:P576 [] }
  MINUS { ?org wdt:P1366 [] }
  MINUS { ?org wdt:P582 [] .
          MINUS { ?org wdt:P582 ?ddate ;
                       wdt:P580 ?idate .
                  FILTER ( ?idate > ?ddate )
               }
        }
  MINUS { 
    ?org p:P31 ?inStatement .
    ?inStatement pq:P582 [] .
    MINUS { ?org p:P31 ?p.
            ?p ps:P31 wd:Q2039348; pq:P580 ?start .
            FILTER NOT EXISTS { ?p pq:P582 []}
    }             
  }
  MINUS { ?org p:P31 ?p.
            ?p ps:P31 wd:Q2039348; pq:P580 ?start .
            FILTER ( ?start > NOW() )
  }

  BIND(REPLACE(STR(?org), "http://www.wikidata.org/entity/", "") AS ?qid)

  OPTIONAL { ?org rdfs:label ?orgLabelEn FILTER(LANG(?orgLabelEn) = "en") }
  OPTIONAL { ?org rdfs:label ?orgLabelNl FILTER(LANG(?orgLabelNl) = "nl") }
  OPTIONAL { ?org rdfs:label ?orgLabelMul FILTER(LANG(?orgLabelMul) = "mul") }
  BIND(COALESCE(?orgLabelEn, ?orgLabelNl, ?orgLabelMul, STR(?org)) AS ?orgLabel)

  OPTIONAL { ?org schema:description ?orgDescriptionEn FILTER(LANG(?orgDescriptionEn) = "en") }
  OPTIONAL { ?org schema:description ?orgDescriptionNl FILTER(LANG(?orgDescriptionNl) = "nl") }
  OPTIONAL { ?org schema:description ?orgDescriptionMul FILTER(LANG(?orgDescriptionMul) = "mul") }
  BIND(COALESCE(?orgDescriptionEn, ?orgDescriptionNl, ?orgDescriptionMul) AS ?orgDescription)

  OPTIONAL { ?type rdfs:label ?typeLabelEn FILTER(LANG(?typeLabelEn) = "en") }
  OPTIONAL { ?type rdfs:label ?typeLabelNl FILTER(LANG(?typeLabelNl) = "nl") }
  OPTIONAL { ?type rdfs:label ?typeLabelMul FILTER(LANG(?typeLabelMul) = "mul") }
  BIND(COALESCE(?typeLabel2, ?typeLabelEn, ?typeLabelNl, ?typeLabelMul, STR(?type)) AS ?typeLabel)
}
ORDER BY ?type ?typeLabel ?orgLabel

RVA2869 · 2026-09-14T17:32:40Z

Could you also add these changes to ADD_A_COUNTRY.md?

fenymufyd · 2026-09-15T13:44:26Z

Could you also add these changes to ADD_A_COUNTRY.md?

I've changed the markdown tutorials too at commit 552930f

Ainali · 2026-10-19T08:50:09Z

In the queries for each country, I wonder why for orgLabel and typeLabel the COALESCE falls back to STR of each, but the description is not?

fenymufyd · 2026-10-22T07:01:02Z

In the queries for each country, I wonder why for orgLabel and typeLabel the COALESCE falls back to STR of each, but the description is not?

Correct me if I'm wrong but I believe it is a kind of a fallback mechanism where if there is no both ?orgLabelEn, ?orgLabelMul values on an item, STR(?org) will output the wikidata QID entity link as ?orgLabel value instead of no value. The same thing applies with ?typeLabel. But with description, STR(?orgDescription) will always output no value so it is useless to add it there

You can try by disabling each OPTIONAL rdfs:label lines to see how STR(?...) works in orgLabel, orgDescription and typeLabel

Ainali · 2026-10-22T07:56:17Z

Now I remember why we don't have it, it wou literally be the same qid as for the label.

…

On Thu, Oct 22, 2026, 09:01 fymd ***@***.***> wrote: *fenymufyd* left a comment (govdirectory/website#661) <#661 (comment)> In the queries for each country, I wonder why for orgLabel and typeLabel the COALESCE falls back to STR of each, but the description is not? Correct me if I'm wrong but I believe it is a kind of a fallback mechanism where if there is no both ?orgLabelEn, ?orgLabelMul values on an item, STR(?org) will output the wikidata QID entity link as ?orgLabel value instead of no value. The same thing applies with ?typeLabel. But with description, STR(?orgDescription) will always output no value so it is useless to add it there You can try by disabling each OPTIONAL rdfs:label lines to see how STR(?...) works in orgLabel, orgDescription and typeLabel — Reply to this email directly, view it on GitHub <#661?email_source=notifications&email_token=ABKO5T3TDHI7KSGJRIQNUBT5VGWEJA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMZTGQ4DCOJVGE22M4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-6334819515>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKO5T4ZKIXGZZSHTRTNRGT5VGWEJAVCNFSNUABFKJSXA33TNF2G64TZHMZTMNRUGEZDSNJVHNEXG43VMU5TIOBZGI4DGMBXGUZKC5QC> . You are receiving this because you commented.Message ID: ***@***.***>

Abbe98 · 2026-11-04T10:12:27Z

Sorry about the time it’s taking to get this merged, it’s still very much on the radar and prioritized.

fenymufyd added 5 commits July 15, 2026 02:41

port wikibase:label queries for qlever compatibility

7250671

port wikibase:label queries for qlever compatibility

552930f

port wikibase:label queries for qlever compatibility

593d0e7

port wikibase:label and SAMPLE in queries for qlever compatibility

6ef17e3

optimized query and port SAMPLE and wikibase:label

d35011f

Abbe98 self-requested a review July 15, 2026 13:50

Abbe98 requested changes Jul 15, 2026

View reviewed changes

Comment thread queries/generators/ukraine.rq Resolved

Comment thread queries/countries.rq Resolved

Comment thread queries/countries.rq Resolved

fenymufyd commented Jul 15, 2026

View reviewed changes

Comment thread queries/organization-optional.rq Resolved

fenymufyd commented Jul 15, 2026

View reviewed changes

Abbe98 added 2 commits July 17, 2026 15:03

Qlever support: special binding for typeLabel in Sweden query

54b577a

Add comment for clarity in organization-optional query

43f3927

Added a comment to clarify the purpose of the OPTIONAL block.

Merge branch 'main' into qlever-compatible-query-port

f6f3d26

fix netherlands.rq for qlever compatibility

a75928e

fenymufyd requested a review from Abbe98 September 15, 2026 13:46

Merge branch 'main' into qlever-compatible-query-port

6734d09

Conversation

fenymufyd commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue Reference

Checklist

Uh oh!

Abbe98 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fenymufyd left a comment

Choose a reason for hiding this comment

Uh oh!

Abbe98 commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Abbe98 commented Jul 17, 2026

Uh oh!

Abbe98 commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fenymufyd commented Sep 14, 2026

Uh oh!

RVA2869 commented Sep 14, 2026

Uh oh!

fenymufyd commented Sep 15, 2026

Uh oh!

Ainali commented Oct 19, 2026

Uh oh!

fenymufyd commented Oct 22, 2026

Uh oh!

Ainali commented Oct 22, 2026 via email

Uh oh!

Abbe98 commented Nov 4, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fenymufyd commented Jul 15, 2026 •

edited

Loading

Abbe98 commented Jul 15, 2026 •

edited

Loading

Abbe98 commented Sep 2, 2026 •

edited

Loading