brightsky in Arduino/ESP8266 environment

[MHz000]

Thank you Jdemaeyer for the great undertaking of providing the DWD weather data in a user-friendly format.
However, I'm running into a problem retrieving the data from my "hobby" environment. When I contact the brightsky server I get a 0-string in response :-((
Who is responsible? My program or the server? If I query the "https://www.howsmyssl.com/a/check" server with my program I get a correct answer. On the other hand, I believe that you and others have tested Brightsky very carefully.
It would be great if you or one of your colleagues could test my program. I am attaching the code. It would be just as good if someone would post a working example for the Arduino/ESP8266. I'm definitely not the only one who is interested in this.

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClientSecureBearSSL.h>   //copy from github 

// Replace with your network credentials
const char* ssid = "***";
const char* password = "***";

void setup() {
  Serial.begin(115200);
  //Serial.setDebugOutput(true);

  Serial.println();
  Serial.println();
  Serial.println();

  //Connect to Wi-Fi
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.println("Connecting to WiFi ..");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print('.');
    delay(1000);
  }
  Serial.println();
}

void loop() {
  // wait for WiFi connection
  if ((WiFi.status() == WL_CONNECTED)) {

    std::unique_ptr<BearSSL::WiFiClientSecure>client(new BearSSL::WiFiClientSecure);

    // Ignore SSL certificate validation
    client->setInsecure();
    
    //create an HTTPClient instance
    HTTPClient https;
    
    //Initializing an HTTPS communication using the secure client
    Serial.print("[HTTPS] begin...\n");
    //if (https.begin(*client, "https://www.howsmyssl.com/a/check")) {  // <<< this request returns a string>>>
    if (https.begin(*client, "https://api.brightsky.dev/weather?lat=52&lon=7.6&date=2020-04-21")) {  // <<< this one not  
      Serial.print("[HTTPS] GET...\n");
      // start connection and send HTTP header
      int httpCode = https.GET();
        // HTTP header has been send and Server response header has been handled
        Serial.printf("[HTTPS] GET... code: %d\n", httpCode);
      // httpCode will be negative on error
      if (httpCode > 0) {
        // file found at server
        if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
          String payload = https.getString();
          //Serial.println(payload.length());
          Serial.printf("payload length is %i\n", payload.length());
          Serial.println(payload);
        }
      } else {
        Serial.printf("[HTTPS] GET... failed, error: %s\n", https.errorToString(httpCode).c_str());
      }
      https.end();
    } else {
      Serial.printf("[HTTPS] Unable to connect\n");
    }
  }
  Serial.println();
  Serial.println("Waiting 2min before the next round...");
 //while(true){yield(); delay(100);};  
  delay(120000);
}

Thank you

[jdemaeyer]

Hi @MHz000, thanks for reaching out! I edited your post to fix the code highlighting.

Could you please paste the output from your Serial Monitor? Additionally, could you please try with this URL (which returns less data): https://api.brightsky.dev/current_weather?lat=52&lon=7.6?

Thanks!

[MHz000]

Hi jdemaeyer,
the browser response:
Mit dieser XML-Datei sind anscheinend keine Style-Informationen verknüpft. Nachfolgend wird die Baum-Ansicht des Dokuments angezeigt.
answer:

<title>Invalid parameter</title> The "lon" parameter is invalid. The value must be a float.

my program output (original URL):
Connecting to WiFi ..
....
[HTTPS] begin...
[HTTPS] GET...
[HTTPS] GET... code: 200
payload length is 0

Waiting 2min before the next round...

thanks for fast response

[jdemaeyer]

Sorry, that was badly formatted on my part. The question mark is not part of the URL, it should end with lon=7.6, so that the URL reads:
https://api.brightsky.dev/current_weather?lat=52&lon=7.6

Could you add the following code directly above String payload = ...:

          Serial.println("content-length: %i\n", https.getSize());

Then retry running the code both with your original URL and with the new URL, and paste the serial monitor content?

[MHz000]

Hi Jakob,
Serial.printf("content-length: %i\n", https.getSize()); has taken us one step further
Output with the original URL:
[HTTPS] GET...
[HTTPS] GET... code: 200
payload length is 0
content-length: 11450 <<<<<<<<<<<<<<<< :-))

While
Serial.printf("payload length is %i\n", payload.length());
still outputs 0 and
Serial.println(payload);
no output produced.
It's definitely due to my program. But what am I doing wrong?
thank you for your patience and support

[jdemaeyer]

Could you please please try with the other URL to see if this is related to the response size? ;)

[MHz000]

https://api.brightsky.dev/current_weather?lat=52&lon=7.6
prouce
....
[HTTPS] begin...
[HTTPS] GET...
[HTTPS] GET... code: 200
payload length is 989
content-length: 989
{"weather":{"source_id":238685,"timestamp":"2024-01-08T15:30:00+00:00","cloud_cover":0,"condition":"dry","dew_point":-9.54,"solar_10":0.001,"solar_30":0.006,"solar_60":0.031,"precipitation_10":0.0,"precipitation_30":0.0,"precipitation_60":0.0,"pressure_msl":1033.5,"relative_humidity":56,"visibility":53041,"wind_direction_10":60,"wind_direction_30":57,"wind_direction_60":55,"wind_speed_10":15.8,"wind_speed_30":15.5,"wind_speed_60":15.8,"wind_gust_direction_10":50,"wind_gust_direction_30":50,"wind_gust_direction_60":50,"wind_gust_speed_10":32.0,"wind_gust_speed_30":32.0,"wind_gust_speed_60":33.5,"sunshine_30":15.0,"sunshine_60":45.0,"temperature":-2.0,"fallback_source_ids":{},"icon":"clear-day"},"sources":[{"id":238685,"dwd_station_id":"01766","observation_type":"synop","lat":52.1344,"lon":7.69686,"height":47.8,"station_name":"Muenster/Osnabrueck","wmo_station_id":"10315","first_record":"2024-01-07T09:30:00+00:00","last_record":"2024-01-08T15:30:00+00:00","distance":16364.0}]}

and correct payload.length as well payload output

:-))

[jdemaeyer]

My guess is that there is not enough memory available to fit the response (see esp8266/Arduino#4603).

This thread on the Arduino forum might be of help to you: https://forum.arduino.cc/t/https-getstring-returns-empty-string/1148362

I'm closing this as it's not a Bright Sky issue, but feel free to keep posting. :) Note that there is a related discussion about adding Bright Sky parameters to reduce the response size: #135

[MHz000]

Thank you Jakob for your commitment to solving Arduino problems too.
If I see it correctly then the Brightsky output is limited to the current weather situation by omitting the date parameter. As suggested, a second way to get around the storage problem would be to only send back the directly requested weather values. Or even easier: use an ESP32.
Thank you very much Jacob

[jdemaeyer]

No sweat at all, my pleasure :)

A small correction: The alternative URL I gave you does not (only) omit the data parameter, but it is a whole different endpoint (/current_weather instead of /weather).

Probably the simplest solution (for now) if you're looking for past or forecasted weather (i.e., if you want to use the /weather endpoint) would be to reduce the number of records requested by using both the date and last_date parameters to create a narrow timestamp range, e.g. https://api.brightsky.dev/weather?lat=52&lon=7.6&date=2020-04-21T12:00:00&last_date=2020-04-21T13:00:00