[Unicon-group] Unicon and native C

[b***@vivaldi.net]

I've been experimenting with ways to call C and COBOL functions without need for a wrapper.

Tests are working out well with (a very platform specific) inline assembly layer.

The test COBOL

identification division.
program-id. sample.

data division.
linkage section.
01 one usage binary-long.
01 two usage binary-long.

procedure division using by value one two.
sample-main.
display "GnuCOBOL got " one ", " two
compute return-code = one + two
goback.
end program sample.

And the test Unicon

#
# testcob.icn, test calling COBOL without wrapper
#
$include "natives.inc"

procedure main()
# will be RTLD_LAZY | RTLD_GLOBAL (so add to the search path)
addLibrary := loadfunc("./uninative.so", "addLibrary")

# allow arbitrary C functions, marshalled by a small piece of assembler
native := loadfunc("./uninative.so", "native")

# add the testing functions to the dlsym search path,
# the handle is somewhat irrelevant, but won't be soonish
dlHandle := addLibrary("./cobolnative.so")

# initialize GnuCOBOL
native("cob_init", TYPEVOID)

# pass two integers, get back a sum
ans := native("sample", TYPEINT, 40, 2)
write("Unicon: called sample and got ", ans)

# rundown the libcob runtime
native("cob_tidy", TYPEVOID)
end

Result pass

prompt$ cobc -m cobolnative.cob
prompt$ unicon -s testcob.icn -x
GnuCOBOL got +0000000040, +0000000002
Unicon: called sample and got 42
Sad part, the inline assembly is only x86_64 System V ABI at the moment. (about 70 lines of inline extended assembler code, some 17 instructions, to manage both integral and double sized reals, input (freely mixed types) and return). That code needs to change to handle floats (versus double) another 17 instructions, without any refactoring. At this point in time, a single function can't have mixed C float and double data, all other type mixes are testing well, but this is only a few hours in to the experiments. Somewhat cherry picked testing, but integer, pointer/handle, real, and string data is functional. There will be edge cases to find and work out, and details regarding memory ownership and the like. This builds on the existing loadfunc feature but most details are then managed by a single entry point, "native(...)".

Windows 64 and 32 and other operating environments should be in the same range of instruction counts, but I'll know a little more once 32bit GNU/Linux is tackled, with the different call frame rules. It might be upwards of 30 instructions that need to be hand coded, not an overly burdensome burden for the powers that it provides to Unicon, in my biased opinion. No wrappers required to get at C library functions.

Jafar, Clinton; I need a little education on the new mkExternal feature in icall.h, as I could not get a working sample. Currently, pointer data is just sent back as an integer handle (which will break on some platforms), and I'd like to extend the supported types to C structs and pointers to memory blocks, etc. I hope that just a short working example on what the actual C data type declaration is to start running with it; could not tell from the macro, so I thought I'd just ask instead of flailing about.

More at https://sourceforge.net/p/unicon/discussion/contributions/thread/4f3103fc/ (https://sourceforge.net/p/unicon/discussion/contributions/thread/4f3103fc/)

And an initial usage sample that embeds libharu for PDF generation at

https://sourceforge.net/p/unicon/discussion/general/thread/5e5d4ab1/ (https://sourceforge.net/p/unicon/discussion/general/thread/5e5d4ab1/)

If this seems like a want-able thing by the Unicon community, it'll require a small piece of assembler for each platform that wants to take part. Getting cross-platform, by writing to each platform. ;-) If that happens, there will be a call out for volunteers that have access to various C compiler versions and OS releases on various chipsets. Expect less than a page of code per platform.

Have good, make well, happiest of 2017s,
Brian

[Don Ward]

Post by b***@vivaldi.net
Jafar, Clinton; I need a little education on the new mkExternal feature in icall.h, as I could not get a working sample. Currently, pointer data is just sent back as an integer handle (which will break on some platforms), and I'd like to extend the supported types to C structs and pointers to memory blocks, etc. I hope that just a short working example on what the actual C data type declaration is to start running with it; could not tell from the macro, so I thought I'd just ask instead of flailing about.

I’m guessing we’re talking about revision 4642 here. If so, the “guilty party” is me. The intention is that an external value is an opaque type: something that is generated by the C code, passed into the Unicon code and given back as required. The Unicon code should not manipulate it or poke around inside it —it’s supposed to be opaque — just give it back to the C code when needed.

A good example of it’s use (which will be coming to a distribution near you fairly soon, I hope) is the way that RFC6234 handles the secure hash context. The context, which is created by the routines, is passed back to the caller, which gives it back to the hash routine for each subsequent operation.

I hope that makes things a bit clearer. Sorry for the confusion, but the RF6234 code, which would have provided an example of how to use mkExternal, is still in the works, awaiting some changes to how things are organised.

Regards

Don

[b***@vivaldi.net]

On Sat, Dec 31, 2016 at 06:52 AM, Don Ward wrote:
On 30 Dec 2016, at 03:34, ***@vivaldi.net (mailto:***@vivaldi.net) wrote:

Jafar, Clinton; I need a little education on the new mkExternal feature in icall.h, as I could not get a working sample. Currently, pointer data is just sent back as an integer handle (which will break on some platforms), and I'd like to extend the supported types to C structs and pointers to memory blocks, etc. I hope that just a short working example on what the actual C data type declaration is to start running with it; could not tell from the macro, so I thought I'd just ask instead of flailing about.

I’m guessing we’re talking about revision 4642 here. If so, the “guilty party” is me. The intention is that an external value is an opaque type: something that is generated by the C code, passed into the Unicon code and given back as required. The Unicon code should not manipulate it or poke around inside it —it’s supposed to be opaque — just give it back to the C code when needed.
A good example of it’s use (which will be coming to a distribution near you fairly soon, I hope) is the way that RFC6234 handles the secure hash context. The context, which is created by the routines, is passed back to the caller, which gives it back to the hash routine for each subsequent operation.
I hope that makes things a bit clearer. Sorry for the confusion, but the RF6234 code, which would have provided an example of how to use mkExternal, is still in the works, awaiting some changes to how things are organised.
Regards
Don

Thanks, Don.

I'm starting to get my head around it a little more. It will come in handy on the next update of uniffi, as I'll be adding struct data passing and return types.

It took a little while to unsquirrel the list management, and I'll be asking for feedback on how that might be improved on, and strengthened. I had to add the b_elem struct to my local copy of icall.h to make it easier on the brain, but then stumbled into how IListVal and RListVal works, and went from there. It is simpler now, then when I was flailing, but I'm not overly sure it's safer or the most efficient.

libffi and calling C routines without loadfunc is pretty handy (and much fun). The libffi cut has a lot of platforms covered, including s390 z/Linux, and Windows and Mac, and FreeBSD and on and on. Making for some pretty neat experiments.

Listings are up at, http://btiffin.users.sourceforge.net/up/multilanguage.html#libffi (http://btiffin.users.sourceforge.net/up/multilanguage.html#libffi)

Still work to do, a lot more tests, adding in all the data subtypes, etc. But it already covers a lot of bases. And not that much code. Worked out the double/float typing problem by using a two element list for arguments that need to be demoted. There is more subtyping to do, and figure out some way of allowing mutable call by content without risking Unicon core types. It'll almost assuredly come down to make temporary or tended copies, and adding another function for accessing the result sets (might be lists, records, encoded strings, whatever structure it ends up looking like for the Unicon programmer side).

Have good, happy 2017 everyone,
Brian

[Jafar Al-Gharaibeh]

We are trying to making it easier for contributors to add/ship extensions
to the language in the form of loadable modules. I've been on the move the
last week and will continue to be away from my development machines until
the second half of January. I will try to make the necessary changes to
the language hopefully before the end of the "new" month.

Cheers,
Jafar