tcl hidden "cruelties"

[aotto1968]

Hi,

I am in the process of rewriting my ALC compiler, whereby the back-end is being rewritten from "tabular" to "object-oriented".

Both types of structures have their advantages and disadvantages. The "tabular" structure is suitable for cross-object analysis
and the "object-oriented" structure is suitable for object-specific analysis.

Unfortunately, you can only ever use ONE type of data structure, so I now use the "tabular" structure in the FRONT-END and the
"object-oriented" structure in the BACK-END.

Anyone who knows TCL knows that "refactoring" TCL code is cruel, because it always ends in a huge amount of "broken" code.

Regardless of "broken code", TCL itself has some cruelties in its syntax. Here, for example, the storage of an "array" with a
namespace path, which in TCL always has !! TWO !! commands. ONE command to generate the namespace and ONE command to finally
generate the array.

namespace eval ::funcDEF::MkErrN {}
array set ::funcDEF::MkErrN::my {
RETURN_MAPPING {}
argv {{ME_CXN_MK_MNGN mng}}
class MkErrorC
classC MkErrorC
classM ME_CCC_MkErrorC
func MkErrN
prefix Class
prefixC ClassC
prefixM ME_CCC_MkErrorC
retC MkErrorC
retM ME_CCN_MkErrorC
retOM ME_CCN_MkErrorC
ty sm_
type0M ME_CXN_MK_MNGN
var0 mng
}

mfg

[Ralf Fassel]

* aotto1968 <***@t-online.de>
| Regardless of "broken code", TCL itself has some cruelties in its
| syntax. Here, for example, the storage of an "array" with a namespace
| path, which in TCL always has !! TWO !! commands. ONE command to
| generate the namespace and ONE command to finally
| generate the array.
| namespace eval ::funcDEF::MkErrN {}
| array set ::funcDEF::MkErrN::my {
| RETURN_MAPPING {}
--<snip-snip>--
| var0 mng
| }

Well, you *could* do it in one step, but you don't have to ;-)

namespace eval ::funcDEF::MkErrN {
variable my
array set my {
RETURN_MAPPING {}
...
var0 mng
}
}

HTH
R'

[Alan Grunwald]

Post by aotto1968
Hi,
I am in the process of rewriting my ALC compiler, whereby the back-end
is being rewritten from "tabular" to "object-oriented".
Both types of structures have their advantages and disadvantages. The
"tabular" structure is suitable for cross-object analysis and the
"object-oriented" structure is suitable for object-specific analysis.
Unfortunately, you can only ever use ONE type of data structure, so I
now use the "tabular" structure in the FRONT-END and the
"object-oriented" structure in the BACK-END.
Anyone who knows TCL knows that "refactoring" TCL code is cruel, because
it always ends in a huge amount of "broken" code.
Regardless of "broken code", TCL itself has some cruelties in its
syntax. Here, for example, the storage of an "array" with a namespace
path, which in TCL always has !! TWO !! commands. ONE command to
generate the namespace and ONE command to finally generate the array.
namespace eval ::funcDEF::MkErrN {}
array set ::funcDEF::MkErrN::my {
    RETURN_MAPPING                                     {}
    argv                                               {{ME_CXN_MK_MNGN
mng}}
    class                                              MkErrorC
    classC                                             MkErrorC
    classM                                             ME_CCC_MkErrorC
    func                                               MkErrN
    prefix                                             Class
    prefixC                                            ClassC
    prefixM                                            ME_CCC_MkErrorC
    retC                                               MkErrorC
    retM                                               ME_CCN_MkErrorC
    retOM                                              ME_CCN_MkErrorC
    ty                                                 sm_
    type0M                                             ME_CXN_MK_MNGN
    var0                                               mng
}
mfg

You are absolutely correct. Choice of data structure, programming style
and programming language is always a compromise. If we are lucky, time
shows that the choices we made were the right ones, for us and for the
job in hand.

Often things prove otherwise, and we are faced with the choice between
reworking everything to change the choices we made previously that we
now view as mistakes, or to accept that some things are more difficult
than they might have been and persevere down the route we chose previously.

As previously we make a choice and in time we can form a view as to
whether we made the right or wrong choice.

In my experience it's never profitable, or helpful, to spend much time
mourning or moaning about the choices we make. Review them, by all
means; if we've made the wrong choice try to determine why we made that
choice and resolve to choose better next time. It's the only way!

besten gruesse

[Gerald Lester]

Post by aotto1968
Hi,
...
Anyone who knows TCL knows that "refactoring" TCL code is cruel, because
it always ends in a huge amount of "broken" code.

Well I guess that is true if your code has lots of hidden dependencies
and is otherwise not well designed in structured.

Post by aotto1968
Regardless of "broken code", TCL itself has some cruelties in its
syntax. Here, for example, the storage of an "array" with a namespace
path, which in TCL always has !! TWO !! commands. ONE command to
generate the namespace and ONE command to finally generate the array.

Ah, I don't know of a single language, including Python, where the
namespace does not have to be "defined" prior to things being "created"
inside of it!

Post by aotto1968
namespace eval ::funcDEF::MkErrN {}
array set ::funcDEF::MkErrN::my {
    RETURN_MAPPING                                     {}
    argv                                               {{ME_CXN_MK_MNGN
mng}}
    class                                              MkErrorC
    classC                                             MkErrorC
    classM                                             ME_CCC_MkErrorC
    func                                               MkErrN
    prefix                                             Class
    prefixC                                            ClassC
    prefixM                                            ME_CCC_MkErrorC
    retC                                               MkErrorC
    retM                                               ME_CCN_MkErrorC
    retOM                                              ME_CCN_MkErrorC
    ty                                                 sm_
    type0M                                             ME_CXN_MK_MNGN
    var0                                               mng
}

As Ralf Fassel noted, you could have done:

namespace eval ::funcDEF::MkErrN {
variable my
array set my {
RETURN_MAPPING {}
...
var0 mng
}
}

[aotto1968]

Post by aotto1968
Hi,
...
Anyone who knows TCL knows that "refactoring" TCL code is cruel, because it always ends in a huge amount of "broken" code.

Well I guess that is true if your code has lots of hidden dependencies and is otherwise not well designed in structured.

Post by aotto1968
Regardless of "broken code", TCL itself has some cruelties in its syntax. Here, for example, the storage of an "array" with a
namespace path, which in TCL always has !! TWO !! commands. ONE command to generate the namespace and ONE command to finally
generate the array.

Ah, I don't know of a single language, including Python, where the namespace does not have to be "defined" prior to things being
"created" inside of it!

Post by aotto1968
namespace eval ::funcDEF::MkErrN {}
array set ::funcDEF::MkErrN::my {
     RETURN_MAPPING                                     {}
     argv                                               {{ME_CXN_MK_MNGN mng}}
     class                                              MkErrorC
     classC                                             MkErrorC
     classM                                             ME_CCC_MkErrorC
     func                                               MkErrN
     prefix                                             Class
     prefixC                                            ClassC
     prefixM                                            ME_CCC_MkErrorC
     retC                                               MkErrorC
     retM                                               ME_CCN_MkErrorC
     retOM                                              ME_CCN_MkErrorC
     ty                                                 sm_
     type0M                                             ME_CXN_MK_MNGN
     var0                                               mng
}

    namespace eval ::funcDEF::MkErrN {
      variable my
      array set my {
          RETURN_MAPPING {}
          ...
          var0 mng
      }
    }

this example has 3 commands

1. namespace eval
2. variable
3. array set

one more than my example

the core question is why the NS is *NOT* created by default.
-> If you can raise an error you can also create the namespace.

the core-issue here is that the "TCL" maintainer wants to protect the programmer for itself...
-> this is usually a "bad" habit because the maintainer try to know better what the programmer wants
as the programmer itself.

just for the background, a "dynamic-namespace" in TCL is something like a "pointer" in C because a "POINTER" open an
"dynamic-namespace" in C. the TCL advantage is that you can choose a *good* name for the pointer, in my case something like
"::class::$instance" and the "::call::$instance::my" is just the instance memory.

the TCL maintainer has a view on the "namespace" feature like a traditional "namespace" in c++ or java etc but lost the
dynamic-pointer view of the "namespace".

[Ralf Fassel]

* aotto1968 <***@t-online.de>
| the core question is why the NS is *NOT* created by default.
| -> If you can raise an error you can also create the namespace.

If you have ever happend to have a namespace tpyo in your 'array set'
command, and did not see it while looking straight at it for half an
hour, you would be grateful for TCL telling you that you mistyped it.

Anyway, this discussion is moot, as I don't think TCL will change in
that respect. I at least would object to auto-creating the namespace :-)

EOD4me...
R'

[aotto1968]

Post by Ralf Fassel
| the core question is why the NS is *NOT* created by default.
| -> If you can raise an error you can also create the namespace.
If you have ever happend to have a namespace tpyo in your 'array set'
command, and did not see it while looking straight at it for half an
hour, you would be grateful for TCL telling you that you mistyped it.
Anyway, this discussion is moot, as I don't think TCL will change in
that respect. I at least would object to auto-creating the namespace :-)
EOD4me...
R'

this is not as trivial as it look like because TCL by default is NON-declaration language.

Post by Ralf Fassel
set myvar 1

always succeed because you don't have to "declare" the "myvar" and just set it

in difference to

Post by Ralf Fassel
set ::myns::myvar

you have to declare the "myns" this is just an extra unnecessary step this make everything more complex without
adding any advantage.

Post by Ralf Fassel
"If you have ever happened to have a namespace typo in your 'array set'"

as I mentioned early → you want to protect the programmer for itself → this is just a bad behavior.

You have to treat the "myns" as part of the name like "myns_myvar" etc in old TCL days without NS

Post by Ralf Fassel
set myns_myvar 1

would never be complied by TCL because the "myns" prefix was just chosen by the programmer.

mfg

[aotto1968]

Post by aotto1968
Hi,
I am in the process of rewriting my ALC compiler, whereby the back-end is being rewritten from "tabular" to "object-oriented".
Both types of structures have their advantages and disadvantages. The "tabular" structure is suitable for cross-object analysis
and the "object-oriented" structure is suitable for object-specific analysis.
Unfortunately, you can only ever use ONE type of data structure, so I now use the "tabular" structure in the FRONT-END and the
"object-oriented" structure in the BACK-END.
Anyone who knows TCL knows that "refactoring" TCL code is cruel, because it always ends in a huge amount of "broken" code.

here we are…

146 files changed, 8351 insertions(+), 6129 deletions(-)

~8351 changes later, the NEW ALC compiler is in duty :-)

https://www.facebook.com/photo/?fbid=853356390326470&set=a.805828751745901

[aotto1968]

git diff --shortstat alc-new-1-lw .

83 files changed, 5214 insertions(+), 5300 deletions(-)


The last step in converting the ALC compiler to the "object" structure has now been completed.

The "attribute database" is the largest and most important database in the ALC architecture because attributes from all other
databases are stored there and thus there is a connection to all other databases. The "object"-specific attributes are now
stored in the data structure of the "object", which means that, in contrast to the "table" structure, the "access path" to the
attribute is now static, which is an *enormous* gain in speed while simplifying the "ALC code".

The ALC compiler consists of three "layers". "frontend", "middleware" and "backend". The "frontend" now uses the "table"
structure and the "backend" uses the "object structure". When converting the "backend" from the "table" to the "object"
structure, hidden "logic problems" came to light that were also eliminated and at the same time it also helps to consolidate the
"layers" themselves because an analysis can only be implemented most efficiently *either* in the "table" structure or only in
the "object" structure, which then *necessarily* triggers a certain reorganization of the ALC code.

The "frontend" layer creates the "meta-code" which:
1. implements a "singular" definition of information
2. is "ordered" in terms of structure, which always delivers the same "logic" "output" for "logic"-like "input".
3. is intended to be stored in the *code repository* in order to ultimately secure the API or to centrally verify a change.
4. is human-readable and changeable in order to quickly create "test data" to create or to add your own "meta-code" extensions.

The "middleware" layer creates the "compiler code" which is used *internally* to control the backend very quickly and
efficiently. In addition, the "middleware" layer has a basic analysis of the "meta-code" and creates a variety of
"access-oriented" data structures from the "singular" "meta-code". The "middleware" layer uses a type of "hibernate" technology
to store a "memory dump" on the hard disk after the successful "meta-code analysis". The ALC compiler can also skip the
"middleware layer" with the command line switch "-force", which then means that the "backend layer" is controlled directly with
the "meta code", which is of course slower but is used for "recursive compilations" (generation of "meta code" from "meta code").

The "backend layer" now creates the desired target, which is then a programming language, a data file or any structured data
storage with a defined format.

https://www.facebook.com/permalink.php?story_fbid=pfbid037cZ3kX4yBNxZNv8NQ6vtpq6nomq6YCWsjhKpPoXbyQ5RXxRrc9YnMAfoa9dAAtvVl&id=100069563501101

[Rich]

Post by aotto1968
Regardless of "broken code", TCL itself has some cruelties in its
syntax. Here, for example, the storage of an "array" with a
namespace path, which in TCL always has !! TWO !! commands. ONE
command to generate the namespace and ONE command to finally generate
the array.
namespace eval ::funcDEF::MkErrN {}
array set ::funcDEF::MkErrN::my {
RETURN_MAPPING {}
...
}

This is Tcl. If something is that bothersome, just morph the language
to be the way you want it to work. I.e.:

proc ns-array-set {fullvarname contents} {
namespace eval [namespace qualifiers $fullvarname] {}
array set $fullvarname $contents
}

Create that once, then use it, instead of plain "array set" whenever
you want to create a namespace, and then set an array within, i.e.:

% ns-array-set abc::pdq::xyz [list a 1 b 2 c 3]

Which has now, in a single command, created the parent namespaces, and
the array variable therein:

% info exists abc::pdq::xyz
1
% parray abc::pdq::xyz
abc::pdq::xyz(a) = 1
abc::pdq::xyz(b) = 2
abc::pdq::xyz(c) = 3
%

And, given that the 'array' command is itself a namespace ensemble, you
could extend the 'array' ensemble to add a "ns-set" (or whatever name
you like) to the ensemble that performs the above "create ns - and
init" of a namespace variable, then you could do
"array ns-set [list a 1 b 2]"
in your code to replace the two commands.

[aotto1968]

Post by Rich

Post by aotto1968
Regardless of "broken code", TCL itself has some cruelties in its
syntax. Here, for example, the storage of an "array" with a
namespace path, which in TCL always has !! TWO !! commands. ONE
command to generate the namespace and ONE command to finally generate
the array.
namespace eval ::funcDEF::MkErrN {}
array set ::funcDEF::MkErrN::my {
RETURN_MAPPING {}
...
}

This is Tcl. If something is that bothersome, just morph the language
proc ns-array-set {fullvarname contents} {
namespace eval [namespace qualifiers $fullvarname] {}
array set $fullvarname $contents
}
Create that once, then use it, instead of plain "array set" whenever
% ns-array-set abc::pdq::xyz [list a 1 b 2 c 3]
Which has now, in a single command, created the parent namespaces, and
% info exists abc::pdq::xyz
1
% parray abc::pdq::xyz
abc::pdq::xyz(a) = 1
abc::pdq::xyz(b) = 2
abc::pdq::xyz(c) = 3
%
And, given that the 'array' command is itself a namespace ensemble, you
could extend the 'array' ensemble to add a "ns-set" (or whatever name
you like) to the ensemble that performs the above "create ns - and
init" of a namespace variable, then you could do
"array ns-set [list a 1 b 2]"
in your code to replace the two commands.

your hint with the "ensemble" feature was good but missing the required code.
to "extend" an ensemble is quite complicate in TCL… follow the code below:

proc ::array-ns-set {fullvarname contents} {
namespace eval [namespace qualifiers $fullvarname] {}
array set $fullvarname $contents
}

namespace ensemble configure array -map [dict replace \
[namespace ensemble configure array -map] ns-set ::array-ns-set]

array ns-set ::x::my [list a 1 b 2]

parray ::x::my


1) but the CORE problem is not solved, for every "namespace ns-set ..." command TWO commands
are executed just to avoid FIRST namespace missing error.
→ (ALL pay runtime-speed just for FIRST benefit)

2) the syntax to extend an existing ensemble is still very "ugly.."

Post by Rich
1. namespace ensemble add array ns-set ::array-ns-set
2. namespace ensemble delete array ns-set

[et99]

Post by aotto1968
1) but the CORE problem is not solved, for every "namespace ns-set ..." command TWO commands
are executed just to avoid FIRST namespace missing error.
→ (ALL pay runtime-speed just for FIRST benefit)
2) the syntax to extend an existing ensemble is still very "ugly.."

1. namespace ensemble add     array ns-set ::array-ns-set
2. namespace ensemble delete  array ns-set

If you have a proposal to improve the TCL language then the TIP mechanism is the approach to take.

However, IMHO, namespace commands, especially ones that create namespaces should be done with the [namespace] command. It would be helpful to be able to grep for "namespace" to find all the relevant commands. Having an exception, where some other command also can create a namespace seems quite odd to me and could cause some confusion for someone other than the author trying to understand or maintain a TCL program.

As to performance, I doubt that a command very likely done but once in a program will cause any issues in that regard.

That said, I wouldn't mind seeing a command:

namespace create name ?script?

which seems to me to be an oversight. When I first looked into namespaces, I found it odd that they were created with the eval option. At minimum, this would be a useful self documenting addition.

-e

[Rich]

Post by aotto1968

Post by Rich

Post by aotto1968
Regardless of "broken code", TCL itself has some cruelties in its
syntax. Here, for example, the storage of an "array" with a
namespace path, which in TCL always has !! TWO !! commands. ONE
command to generate the namespace and ONE command to finally generate
the array.

This is Tcl. If something is that bothersome, just morph the language
proc ns-array-set {fullvarname contents} {
namespace eval [namespace qualifiers $fullvarname] {}
array set $fullvarname $contents
}
Create that once, then use it, instead of plain "array set" ...

your hint with the "ensemble" feature was good but missing the required code.

Yes, I left that exercise to the reader to work out.
As are most /things/ in most programming languages.

Post by aotto1968
1) but the CORE problem is not solved, for every "namespace ns-set
..." command TWO commands are executed just to avoid FIRST namespace
missing error. → (ALL pay runtime-speed just for FIRST benefit)

You'll pay that runtime no matter whether it appears as one, or two,
commands in your source. The 'work' of checking if the namespace
exists, and then creating it when it does not, has to be paid, no
matteer how many 'commands' appear in the Tcl script.

Post by aotto1968
2) the syntax to extend an existing ensemble is still very "ugly.."

Post by Rich
1. namespace ensemble add array ns-set ::array-ns-set
2. namespace ensemble delete array ns-set

You can 'wrap' that ugliness into your own proc, and 'extend' the
namespace ensemble itself, to remove that as well if you like.