Soupgate looked for 0000????.msg Soup files and couldn't open the 0000????.MSG Soup files. This problem was introduced when I changed the extension for Fidonet messages from ".MSG" to ".msg". It exposed a hidden bug, because Soupgate used the Fidonet .msg extension for Soup files instead of the Soup extension which is still defined as ".MSG".

Unfortunately the more I use soupgate the less I like it. It's simple to use, but it's approach to FTN<->Internet message gating is way to simplistic. No real charset conversions, no TZUTC support, dumping the complete RFC822 header into kludge lines (even multi-line fields), non-Gatebau compliant MSGID/Message-IDs. It's the worst case scenario for dupes: changed datetime, changed text body, changed MGSID, a bunch of weird kludges.

I wonder if it is best to look for better alternatives. I guess even a rewrite from scratch would be not much more work then extending and bugfixing Soupgate and Spoon.

I downloaded some of the Gatebau '94 and '97 documentation. Mostly in German, but the most important one is available in English too:
https://fidogate.sourceforge.net/msgid.html

Soupgate does it all wrong ;).

I don't mind doing a bit more bug fixing on Soupgate, but I don't think I want to dive into the "double free" Spoon code. I let Spoon connect to my JamNNTPd server. It didn't even try to login a user (username and password is in the config) and crashed the server and then itself. Isn't it possible to run vSoup in wine? Or is there some other Soup client that can be used on linux?

Btw, I just stumbled upon a Soupgate thread on LINUX_BBS from last year (30 Jun 20 "Soupgate / Spoon"). Same problems.

---
* Origin: . (21:3/102)

On 27 Jan 2021 at 07:13p, Oli pondered and said...

Soupgate looked for 0000????.msg Soup files and couldn't open the 0000????.MSG Soup files. This problem was introduced when I changed the extension for Fidonet messages from ".MSG" to ".msg". It exposed a

Good catch... I also found that with an earlier version when running the
export function Soupgate was looking for .CLO not .clo files...
Perhaps this is the same issue?

Unfortunately the more I use soupgate the less I like it. It's simple to use, but it's approach to FTN<->Internet message gating is way to simplistic. No real charset conversions, no TZUTC support, dumping the complete RFC822 header into kludge lines (even multi-line fields), non-Gatebau compliant MSGID/Message-IDs. It's the worst case scenario
for dupes: changed datetime, changed text body, changed MGSID, a bunch
of weird kludges.

Many kludges can be disabled by config file. I agree it's not perfect but the windows version has been working OK for several years for me without too many niggles on either side of the gate.

I wonder if it is best to look for better alternatives. I guess even a rewrite from scratch would be not much more work then extending and bugfixing Soupgate and Spoon.

One reason I like the combo is that they are two stand alone exec files that
I can install, config and run. It doesn't require a huge overhead to install something more complex.. But I agree fixing bugs and extending features would be great. Just wish I had the coding skills to do it :(

I don't mind doing a bit more bug fixing on Soupgate, but I don't think
I want to dive into the "double free" Spoon code. I let Spoon connect to my JamNNTPd server. It didn't even try to login a user (username and password is in the config) and crashed the server and then itself. Isn't it possible to run vSoup in wine? Or is there some other Soup client
that can be used on linux?

It may be the way spoon talks to your NNTP is different to the INN NNTP I am using. I know the later compiled version we have got working to pull down
news seems to work alright and I can post to NNTP which does require a
username and password on my system.. and that works also... it's just the
hand off at the end when the system borks on the known error that is an issue

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-28):

On 27 Jan 2021 at 07:13p, Oli pondered and said...

Soupgate looked for 0000????.msg Soup files and couldn't open the
0000????.MSG Soup files. This problem was introduced when I changed
the extension for Fidonet messages from ".MSG" to ".msg". It
exposed a

Good catch... I also found that with an earlier version when running the export function Soupgate was looking for .CLO not .clo files...
Perhaps this is the same issue?

Yes and no. It's defined in soupgate.c (and can be changed easily):

const char binkmailext[BINK_ALLTYPES][5]
= {".out",".cut",".dut",".hut"};

const char binkflowext[BINK_ALLTYPES][5]
= {".flo",".clo",".dlo",".hlo"};

const char packetext[5]
= ".pkt";

const char messageext[5]
= ".msg";

const char soupmsgext[5]
= ".MSG";

The problem with "soupgate import" was that it used messageext, which is the extension for ArcMail (netmail) messages. The extension for the Soup files is defined by soupmsgext, but never was used on import. As long as both were ".MSG" in uppercase it didn't make a difference, but when I changed messageext from ".MSG" to ".msg" it didn't work as expected anymore.

Now it's fixed and they should work independent from each other.

Unfortunately the more I use soupgate the less I like it. It's
simple to use, but it's approach to FTN<->Internet message gating
is way to simplistic. No real charset conversions, no TZUTC
support, dumping the complete RFC822 header into kludge lines (even
multi-line fields), non-Gatebau compliant MSGID/Message-IDs. It's
the worst case scenario for dupes: changed datetime, changed text
body, changed MGSID, a bunch of weird kludges.

Many kludges can be disabled by config file. I agree it's not perfect but the windows version has been working OK for several years for me without too many niggles on either side of the gate.

There are standards and best practices for a reason. It might work fine for you now, but in another scenario it might generate all kinds of weird side-effects and problems. Like the Mystic tosser with all it's non-standard behavior worked okay, until it didn't (AFAIK the various bugs have been fixed).

Soupgate is creating a real mess at re-importing and translating MSGID/REPLYs. One problem is that even for local use reply linking cannot work, when MSGID / Message-IDs aren't translated transparently back and forth.

Btw, I don't think binkp.net is meant to be used for mail/news gating and that fidonet.org would still be the better (only correct?) option for the Message-ID domain.

I wonder if it is best to look for better alternatives. I guess
even a rewrite from scratch would be not much more work then
extending and bugfixing Soupgate and Spoon.

One reason I like the combo is that they are two stand alone exec files that I can install, config and run. It doesn't require a huge overhead to install something more complex..

That's what I liked with soupgate too ...

But I agree fixing bugs and extending
features would be great. Just wish I had the coding skills to do it :(

.... but then it's maybe more work (for us) to get it in a good state than learning to configure a complex monster like fidogate.

---
* Origin: . (21:3/102)

On 28 Jan 2021 at 02:42p, Oli pondered and said...

changed messageext from ".MSG" to ".msg" it didn't work as expected anymore.

Now it's fixed and they should work independent from each other.

Thanks for this explanation and some insights as to how that code works.

There are standards and best practices for a reason. It might work fine for you now, but in another scenario it might generate all kinds of
weird side-effects and problems.

Fair enough.

Like the Mystic tosser with all it's
non-standard behavior worked okay, until it didn't (AFAIK the various
bugs have been fixed).

I'm not going to get into the pros and cons of Mystic but agree each system
has it's advantages and disadvantages. Running all of fsxNet with Mystic HUBs was fine but when some were not using the same tossing software it became apparent there were differences in the way packets were being treated.

Soupgate is creating a real mess at re-importing and translating MSGID/REPLYs. One problem is that even for local use reply linking
cannot work, when MSGID / Message-IDs aren't translated transparently
back and forth.

I think there may be a setting in the ini to adjust some of this?

Btw, I don't think binkp.net is meant to be used for mail/news gating
and that fidonet.org would still be the better (only correct?) option
for the Message-ID domain.

I'd only used is as I thought the original fidonet domain used that shipped with the initial files had been lost to a domain name squatter. It can be easily adjusted in the ini thankfully.

One reason I like the combo is that they are two stand alone exec file that I can install, config and run. It doesn't require a huge overhead install something more complex..

That's what I liked with soupgate too ...

Simple is good.

But I agree fixing bugs and extending
features would be great. Just wish I had the coding skills to do it :(

.... but then it's maybe more work (for us) to get it in a good state
than learning to configure a complex monster like fidogate.

I've not yet had time to rebuild the new code and test it but will look to do so today. In terms of moving 3:770/1 over to HPT I was wanting to get the gating software running first for Liunx but may well just fudge some shuffle
of packets between Linux and Windows to keep using the software I have
running under Windows for Soupgate and VSoup while a better more long term solution for gating under Linux can be found.

Fidogate (I think) needs to be on the same system as an NNTP server (not
sure) and if so it will be a bit of work for me to setup as I am also wanting to look at migrating my NNTP setup over to the same system that will run the Fido hub... but one thing at a time :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-29):

changed messageext from ".MSG" to ".msg" it didn't work as expected
anymore.

Now it's fixed and they should work independent from each other.

Thanks for this explanation and some insights as to how that code works.

I found the specification for the "Simple Offline USENET Packet Format (SOUP) Version 1.2". For SOUP files it's always upper case: "Each of the filenames in a packet should be stored in upper case on those systems where case matters (e.g. Unix)". So I don't expect any further problems with filenames.

Soupgate is creating a real mess at re-importing and translating
MSGID/REPLYs. One problem is that even for local use reply linking
cannot work, when MSGID / Message-IDs aren't translated
transparently back and forth.

I think there may be a setting in the ini to adjust some of this?

I don't see a setting that would help. This has to be changed in the code.

Btw, I don't think binkp.net is meant to be used for mail/news
gating and that fidonet.org would still be the better (only
correct?) option for the Message-ID domain.

I'd only used is as I thought the original fidonet domain used that
shipped with the initial files had been lost to a domain name squatter.
It can be easily adjusted in the ini thankfully.

I think that was fidonet.net not .org and it is only relevant for DNS (binkp mailers calling each other).

From binkd's HISTORY file:
2011/08/17 12:02:59 1.0a-560 gul
Default root-domain fidonet.net changed to binkp.net

One reason I like the combo is that they are two stand alone
exec file that I can install, config and run. It doesn't
require a huge overhead install something more complex..

That's what I liked with soupgate too ...

Simple is good.

I remember that someone complained about hpt being too complex ... (it wasn't me, was it?).

.... but then it's maybe more work (for us) to get it in a good
state than learning to configure a complex monster like fidogate.

I've not yet had time to rebuild the new code and test it but will look
to do so today.

If you are lucky, soupgate will work as expected. I thought a bit more about soupgate and maybe it's still a good idea to keep it alive.

I also checked out Fidogate and instantly lost all motivation to learn it. Maybe I'll try another day.

Fidogate (I think) needs to be on the same system as an NNTP server (not sure) and if so it will be a bit of work for me to setup as I am also wanting to look at migrating my NNTP setup over to the same system that will run the Fido hub... but one thing at a time :)

Isn't it possible to send it via rnews to an nntpd server on another machine?

https://www.eyrie.org/~eagle/software/inn/docs-2.6/rnews.html

* Origin: . (21:3/102)

On 29 Jan 2021 at 12:49p, Oli pondered and said...

I found the specification for the "Simple Offline USENET Packet Format (SOUP) Version 1.2". For SOUP files it's always upper case: "Each of the filenames in a packet should be stored in upper case on those systems where case matters (e.g. Unix)". So I don't expect any further problems with filenames.

interesting, thanks for the details... yep I concur.

I think that was fidonet.net not .org and it is only relevant for DNS (binkp mailers calling each other).

ah

I remember that someone complained about hpt being too complex ... (it wasn't me, was it?).

Hahah, we'll I'm getting better at HPT and soon to throw a switch or two on that software.

I've not yet had time to rebuild the new code and test it but will loo to do so today.

If you are lucky, soupgate will work as expected. I thought a bit more about soupgate and maybe it's still a good idea to keep it alive.

Tests last night seemed to be fine, import and export, it's only Soup and getting stuff posted back out to NNTP successfully that now seems to be the issue.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)