[Lazarus] fpWeb and server-sent events

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
Hello,

I need to embed a simple web server in my application that

1) should server a couple of static files
2) send server-sent events

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events


I can see that 1) is easy, but 2) I cannot see how (fpWeb model doesn't
seem to allow to take over the socket and keep sending data on it).
I saw an old message regarding websockets and bauglir-websocket but it
seems it implements a different server and I'd like to run everything
from the same socket.

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list


On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:

> Hello,
>
> I need to embed a simple web server in my application that
>
> 1) should server a couple of static files
> 2) send server-sent events
>
> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
>
>
> I can see that 1) is easy, but 2) I cannot see how (fpWeb model doesn't
> seem to allow to take over the socket and keep sending data on it).
> I saw an old message regarding websockets and bauglir-websocket but it
> seems it implements a different server and I'd like to run everything
> from the same socket.

Simply said: You can't in FPC.

The only way to do this with FPC is using websockets, indeed using
bauglir-websocket. I use it myself like that.

Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:

>
>
> On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:
>
>> Hello,
>>
>> I need to embed a simple web server in my application that
>>
>> 1) should server a couple of static files
>> 2) send server-sent events
>>
>> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events 
>>
>>
>>
>> I can see that 1) is easy, but 2) I cannot see how (fpWeb model
>> doesn't seem to allow to take over the socket and keep sending data on
>> it).
>> I saw an old message regarding websockets and bauglir-websocket but it
>> seems it implements a different server and I'd like to run everything
>> from the same socket.
>
> Simply said: You can't in FPC.
>
> The only way to do this with FPC is using websockets, indeed using
> bauglir-websocket. I use it myself like that.

I didn't look into it too much, but does it also serve "normal" http? If
so I could adapt it to manage events (after all it's much simpler than
websockets).

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 3/6/20 a les 16:32, Luca Olivetti via lazarus ha escrit:

> El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:
>>
>>
>> On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:
>>
>>> Hello,
>>>
>>> I need to embed a simple web server in my application that
>>>
>>> 1) should server a couple of static files
>>> 2) send server-sent events
>>>
>>> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events 
>>>
>>>
>>>
>>> I can see that 1) is easy, but 2) I cannot see how (fpWeb model
>>> doesn't seem to allow to take over the socket and keep sending data
>>> on it).
>>> I saw an old message regarding websockets and bauglir-websocket but
>>> it seems it implements a different server and I'd like to run
>>> everything from the same socket.
>>
>> Simply said: You can't in FPC.
>>
>> The only way to do this with FPC is using websockets, indeed using
>> bauglir-websocket. I use it myself like that.
>
> I didn't look into it too much, but does it also serve "normal" http? If
> so I could adapt it to manage events (after all it's much simpler than
> websockets).


I see that I can connect the EventSource to a different socket, so I can
use fpWeb to serve static content and a simple socket to send events.

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
In reply to this post by Free Pascal - Lazarus mailing list


On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:

> El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:
>>
>>
>> On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:
>>
>>> Hello,
>>>
>>> I need to embed a simple web server in my application that
>>>
>>> 1) should server a couple of static files
>>> 2) send server-sent events
>>>
>>>
> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events 
>>>
>>>
>>>
>>> I can see that 1) is easy, but 2) I cannot see how (fpWeb model
>>> doesn't seem to allow to take over the socket and keep sending data on
>>> it).
>>> I saw an old message regarding websockets and bauglir-websocket but it
>>> seems it implements a different server and I'd like to run everything
>>> from the same socket.
>>
>> Simply said: You can't in FPC.
>>
>> The only way to do this with FPC is using websockets, indeed using
>> bauglir-websocket. I use it myself like that.
>
> I didn't look into it too much, but does it also serve "normal" http? If
> so I could adapt it to manage events (after all it's much simpler than
> websockets).

No, it does not serve normal http.


Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
In reply to this post by Free Pascal - Lazarus mailing list
Hi Luca.

I hope you are fine in the current world situation.

Some answers below to your questions 1 and 2.

On Wed, Jun 3, 2020 at 10:30 AM Luca Olivetti via lazarus <[hidden email]> wrote:
Hello,

I need to embed a simple web server in my application that

1) should server a couple of static files
2) send server-sent events
 
1) So, you can use any Pascal structure to serve files, even in a single routine. BTW, if you want to resolve media types, you can use fpmimetypes or BrookMediaTypes.

2) As Michael explained, unfortunately, it is not supported in FCL-Web (yet) . However, if you want a SSE structure out-of-the-box, you can provide it using the Brook streaming, e.g.:


Both examples above declare a small HTML client showing how to consume it via event source (EventSource). Optionally, you can use some async structure (e.g.: TSpinWait.SpinUntil) to create a small polling to push/pop the messages provided to the clients in a single long lived event.

HTH

--
Silvio Clécio

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 3/6/20 a les 18:53, silvioprog via lazarus ha escrit:

> 2) As Michael explained, unfortunately, it is not supported in FCL-Web
> (yet) . However, if you want a SSE structure out-of-the-box, you can
> provide it using the Brook streaming, e.g.:
>
> Minimal console example:
> https://github.com/risoflora/brookframework/blob/expr/Examples/Console/FPC/httpsrvsse.lpr 
>
> Minimal LCL example:
> https://github.com/risoflora/brookframework/blob/expr/Examples/LCL/HTTPServerSSE_frMain.pas

Thank you.
I tested the 2 ports approach (fpweb on one port and a custom socket for
SSE on a different port) and it works, but this is nicer.
Is brookframework stable enough to run 24/7?

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
On Wed, Jun 3, 2020 at 2:04 PM Luca Olivetti via lazarus <[hidden email]> wrote:
El 3/6/20 a les 18:53, silvioprog via lazarus ha escrit:
Thank you.
I tested the 2 ports approach (fpweb on one port and a custom socket for
SSE on a different port) and it works, but this is nicer.

Pretty good! Maybe providing a small example would be useful for the community. ☺

Is brookframework stable enough to run 24/7?

Yes, it is. We have been using it as Apache/Nginx replacement running in production in an application as service (daemon) 24/7 in ~47 customers providing data for several clients (web, mobile and desktop).

--
Silvio Clécio

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 3/6/20 a les 19:41, silvioprog via lazarus ha escrit:

> On Wed, Jun 3, 2020 at 2:04 PM Luca Olivetti via lazarus
> <[hidden email] <mailto:[hidden email]>>
> wrote:
>
>     El 3/6/20 a les 18:53, silvioprog via lazarus ha escrit:
>     Thank you.
>     I tested the 2 ports approach (fpweb on one port and a custom socket
>     for
>     SSE on a different port) and it works, but this is nicer.
>
>
> Pretty good! Maybe providing a small example would be useful for the
> community. ☺

Too quick and dirty to be an example ;-) Maybe tomorrow I'll clean it up
a bit, or maybe I'll go with brookframework directly :-D

>
>     Is brookframework stable enough to run 24/7?
>
>
> Yes, it is. We have been using it as Apache/Nginx replacement running in
> production in an application as service (daemon) 24/7 in ~47 customers
> providing data for several clients (web, mobile and desktop).

Good, then I guess the question is if firexox and/or chromium are stable
enough to run 24/7 (which I doubt...)

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
In reply to this post by Free Pascal - Lazarus mailing list
El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:

>
>
> On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:
>
>> Hello,
>>
>> I need to embed a simple web server in my application that
>>
>> 1) should server a couple of static files
>> 2) send server-sent events
>>
>> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events 
>>
>>
>>
>> I can see that 1) is easy, but 2) I cannot see how (fpWeb model
>> doesn't seem to allow to take over the socket and keep sending data on
>> it).
>> I saw an old message regarding websockets and bauglir-websocket but it
>> seems it implements a different server and I'd like to run everything
>> from the same socket.
>
> Simply said: You can't in FPC.

It turns out I can with some careful, or reckless ;-),  overriding.

https://paste.ubuntu.com/p/CdHfc7BTHB/

Bye

--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list


On Thu, 4 Jun 2020, Luca Olivetti via lazarus wrote:

> El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:
>>
>>
>> On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote:
>>
>>> Hello,
>>>
>>> I need to embed a simple web server in my application that
>>>
>>> 1) should server a couple of static files
>>> 2) send server-sent events
>>>
>>>
> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events 
>>>
>>>
>>>
>>> I can see that 1) is easy, but 2) I cannot see how (fpWeb model
>>> doesn't seem to allow to take over the socket and keep sending data on
>>> it).
>>> I saw an old message regarding websockets and bauglir-websocket but it
>>> seems it implements a different server and I'd like to run everything
>>> from the same socket.
>>
>> Simply said: You can't in FPC.
>
> It turns out I can with some careful, or reckless ;-),  overriding.
>
> https://paste.ubuntu.com/p/CdHfc7BTHB/

Haha, very nice job :-)

I'll see if I can integrate this in the HTTPServer request/response.

It won't work in fastCGI/CGI of course, but if we can make it available in a
http application, why not...


Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 4/6/20 a les 11:48, Michael Van Canneyt ha escrit:

>> It turns out I can with some careful, or reckless ;-),  overriding.
>>
>> https://paste.ubuntu.com/p/CdHfc7BTHB/
>
> Haha, very nice job :-)
>
> I'll see if I can integrate this in the HTTPServer request/response.
>
> It won't work in fastCGI/CGI of course, but if we can make it available
> in a
> http application, why not...


I think that approach (take over the socket from the response) could be
useful in more cases, e.g. streaming content, websockets, etc.
As you say it has to be a stand-alone/embedded server and probably it
only works when it's threaded (due to the connection being kept active
for an indeterminate time), but that's better than nothing.

Now, I have another problem: if the server cannot bind to the port it
throws an exception, but then I cannot try again, I have to destroy and
recreate the server, e.g. this doesn't work (the second call to
FServer.Run return immediately):


   FServer:=TSseServerHandler.Create(nil);
   FServer.Port:=8080;
   FServer.Threaded:=true;
   while not terminated do
   try
     FServer.Run;
   except
     on E:Exception do
     begin
       writeln(E.Message);
       sleep(1000);
       FServer.Terminate;
     end;
   end;


this does (but I don't like it)

   while not terminated do
   try
     FServer:=TSseServerHandler.Create(nil);
     FServer.Port:=8080;
     FServer.Threaded:=true;
     FServer.Run;
   except
     on E:Exception do
     begin
       writeln(E.Message);
       sleep(1000);
       FServer.Free;
     end;
   end;


Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 4/6/20 a les 12:04, Luca Olivetti via lazarus ha escrit:

> El 4/6/20 a les 11:48, Michael Van Canneyt ha escrit:
>
>>> It turns out I can with some careful, or reckless ;-),  overriding.
>>>
>>> https://paste.ubuntu.com/p/CdHfc7BTHB/
>>
>> Haha, very nice job :-)
>>
>> I'll see if I can integrate this in the HTTPServer request/response.
>>
>> It won't work in fastCGI/CGI of course, but if we can make it
>> available in a
>> http application, why not...
>
>
> I think that approach (take over the socket from the response) could be
> useful in more cases, e.g. streaming content, websockets, etc.
> As you say it has to be a stand-alone/embedded server and probably it
> only works when it's threaded (due to the connection being kept active
> for an indeterminate time), but that's better than nothing.
>
> Now, I have another problem: if the server cannot bind to the port it
> throws an exception, but then I cannot try again, I have to destroy and
> recreate the server, e.g. this doesn't work (the second call to
> FServer.Run return immediately):
>
>
>    FServer:=TSseServerHandler.Create(nil);
>    FServer.Port:=8080;
>    FServer.Threaded:=true;
>    while not terminated do
>    try
>      FServer.Run;
>    except
>      on E:Exception do
>      begin
>        writeln(E.Message);
>        sleep(1000);
>        FServer.Terminate;
>      end;
>    end;
>
>
> this does (but I don't like it)
>
>    while not terminated do
>    try
>      FServer:=TSseServerHandler.Create(nil);
>      FServer.Port:=8080;
>      FServer.Threaded:=true;
>      FServer.Run;
>    except
>      on E:Exception do
>      begin
>        writeln(E.Message);
>        sleep(1000);
>        FServer.Free;
>      end;
>    end;

Oh, and I had to do some more overriding so that the response could have
a reference to the calling thread (needed if I want to use synchronize).

Bye

--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list


On Thu, 4 Jun 2020, Luca Olivetti via lazarus wrote:

>>      FServer.Port:=8080;
>>      FServer.Threaded:=true;
>>      FServer.Run;
>>    except
>>      on E:Exception do
>>      begin
>>        writeln(E.Message);
>>        sleep(1000);
>>        FServer.Free;
>>      end;
>>    end;
>
> Oh, and I had to do some more overriding so that the response could have
> a reference to the calling thread (needed if I want to use synchronize).
That seems a bit strange, since you can do synchronize without needing
access to the thread object using a class method of TThread ?

Can you send me the changes you had to do ?

Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 4/6/20 a les 15:37, Michael Van Canneyt via lazarus ha escrit:

>
>
> On Thu, 4 Jun 2020, Luca Olivetti via lazarus wrote:
>
>>>      FServer.Port:=8080;
>>>      FServer.Threaded:=true;
>>>      FServer.Run;
>>>    except
>>>      on E:Exception do
>>>      begin
>>>        writeln(E.Message);
>>>        sleep(1000);
>>>        FServer.Free;
>>>      end;
>>>    end;
>>
>> Oh, and I had to do some more overriding so that the response could
>> have a reference to the calling thread (needed if I want to use
>> synchronize).
>
> That seems a bit strange, since you can do synchronize without needing
> access to the thread object using a class method of TThread ?

TThread.Synchronize needs a TThread as the first parameter.
This is my work in progress

https://paste.ubuntu.com/p/cKdQmCQw7p/

The main difference from the previous pastebin is that I subclass
TFPHTTPConnection to add a TThread field and override the
CreateConnectionThread to add the thread reference to the connection.
And I added more parameters to the server thread to pass on events from
the main thread.


Tlogger is a simple class defined in utils.pas (not included) that logs
messages.

The idea is that every time a new sse connection comes in, I notify the
main thread (not yet implemented), then in DoSendConnect I enter a loop
where I get the data from the main thread (with synchronize) then wait
on an event (that the main thread signals every time the data changes).
If there is no new data, every 10 seconds I send an empty message as a
keepalive.


So, I need synchronize:

  - to notify a new connection
  - to get data for that connection
  - to notify that the connection terminated

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
Luca Olivetti via lazarus <[hidden email]> schrieb am Do., 4. Juni 2020, 15:51:
El 4/6/20 a les 15:37, Michael Van Canneyt via lazarus ha escrit:
>
>
> On Thu, 4 Jun 2020, Luca Olivetti via lazarus wrote:
>
>>>      FServer.Port:=8080;
>>>      FServer.Threaded:=true;
>>>      FServer.Run;
>>>    except
>>>      on E:Exception do
>>>      begin
>>>        writeln(E.Message);
>>>        sleep(1000);
>>>        FServer.Free;
>>>      end;
>>>    end;
>>
>> Oh, and I had to do some more overriding so that the response could
>> have a reference to the calling thread (needed if I want to use
>> synchronize).
>
> That seems a bit strange, since you can do synchronize without needing
> access to the thread object using a class method of TThread ?

TThread.Synchronize needs a TThread as the first parameter.
This is my work in progress

The thread parameter can be Nil. TThread will then use the CurrentThread threadvar and failing that it can also work without any thread object. 

In fact it is safer not to pass a thread. 

Regards, 
Sven 

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
El 4/6/20 a les 17:07, Sven Barth ha escrit:

>
>     TThread.Synchronize needs a TThread as the first parameter.
>     This is my work in progress
>
>
> The thread parameter can be Nil. TThread will then use the CurrentThread
> threadvar and failing that it can also work without any thread object.
>
> In fact it is safer not to pass a thread.

Oh, I didn't know that. Thank you.

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] fpWeb and server-sent events

Free Pascal - Lazarus mailing list
In reply to this post by Free Pascal - Lazarus mailing list
On Wed, Jun 3, 2020 at 3:46 PM Luca Olivetti via lazarus <[hidden email]> wrote:
El 3/6/20 a les 19:41, silvioprog via lazarus ha escrit:
Good, then I guess the question is if firexox and/or chromium are stable
enough to run 24/7 (which I doubt...)

😂 

--
Silvio Clécio

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus