Define an active character inside a definition

Clash Royale CLAN TAG#URR8PPP

up vote
2
down vote

favorite

deftestcatcode`!=active def!test

Of course the above code would not work because nothing is executed when TeX is reading the replacement text of a macro definition. ! gets catcode 12 anyway. I've tried a search on this site and someone suggested using scantokens. As far as I know, scantokens is an expandable command which reads the following token list, stores it in a pseudo-file and then inputs it, so the tokens get re-tokenized according to the new catcodes. However, the following code still fails and I can't figure out why.

deftestcatcode`!=active scantokensdef!test
test

I get the error of "runaway definition".

macros tex-core e-tex

share|improve this question

edited 9 mins ago

Joseph Wright♦

197k21544863

asked 2 hours ago

user5938

1286

add a comment | 

up vote
2
down vote

favorite

deftestcatcode`!=active def!test

Of course the above code would not work because nothing is executed when TeX is reading the replacement text of a macro definition. ! gets catcode 12 anyway. I've tried a search on this site and someone suggested using scantokens. As far as I know, scantokens is an expandable command which reads the following token list, stores it in a pseudo-file and then inputs it, so the tokens get re-tokenized according to the new catcodes. However, the following code still fails and I can't figure out why.

deftestcatcode`!=active scantokensdef!test
test

I get the error of "runaway definition".

macros tex-core e-tex

share|improve this question

edited 9 mins ago

Joseph Wright♦

197k21544863

asked 2 hours ago

user5938

1286

add a comment | 

up vote
2
down vote

favorite

up vote
2
down vote

favorite

deftestcatcode`!=active def!test

Of course the above code would not work because nothing is executed when TeX is reading the replacement text of a macro definition. ! gets catcode 12 anyway. I've tried a search on this site and someone suggested using scantokens. As far as I know, scantokens is an expandable command which reads the following token list, stores it in a pseudo-file and then inputs it, so the tokens get re-tokenized according to the new catcodes. However, the following code still fails and I can't figure out why.

deftestcatcode`!=active scantokensdef!test
test

I get the error of "runaway definition".

macros tex-core e-tex

share|improve this question

edited 9 mins ago

Joseph Wright♦

197k21544863

asked 2 hours ago

user5938

1286

deftestcatcode`!=active def!test

Of course the above code would not work because nothing is executed when TeX is reading the replacement text of a macro definition. ! gets catcode 12 anyway. I've tried a search on this site and someone suggested using scantokens. As far as I know, scantokens is an expandable command which reads the following token list, stores it in a pseudo-file and then inputs it, so the tokens get re-tokenized according to the new catcodes. However, the following code still fails and I can't figure out why.

deftestcatcode`!=active scantokensdef!test
test

I get the error of "runaway definition".

macros tex-core e-tex

macros tex-core e-tex

share|improve this question

edited 9 mins ago

Joseph Wright♦

197k21544863

asked 2 hours ago

user5938

1286

share|improve this question

edited 9 mins ago

Joseph Wright♦

197k21544863

asked 2 hours ago

user5938

1286

share|improve this question

share|improve this question

edited 9 mins ago

Joseph Wright♦

197k21544863

edited 9 mins ago

Joseph Wright♦

197k21544863

edited 9 mins ago

Joseph Wright♦

197k21544863

197k21544863

asked 2 hours ago

user5938

1286

asked 2 hours ago

user5938

1286

asked 2 hours ago

user5938

1286

1286

add a comment | 

add a comment | 

1 Answer
1

active

oldest

votes

up vote
3
down vote



accepted

The e-TeX reference defines scantokens as follows:

scantokens, when followed by a <general text>, decomposes the <balanced text> of the <general text> into the corresponding sequence of characters as if the <balanced text> were written unexpanded to a file; it then uses TeX's input mechanism to re-process these characters under the current catcode regime.

So the def! part of your macro is processed as if it were defined in an extra file that is then input into your main file. When TeX complains about File ended while scanning definition of !, it seems definitions of macros may not span across the end of a file and continue after that. I couldn't find a reference for that anywhere, but you can see the same behavior and error message if you replace scantokensdef! by an actual input were the input file includes only def!.

One way to get the desired result is by defining test in a regime which has the correct catcode already applied:

begingroup
catcode`!=active
gdeftestcatcode`!=active def!test
endgroup
test
show!

Instead of the gdef you could also save the current catcode of ! and restore it after the definition of test.

Another common trick is to change the lowercase code of an active character, here ~, to the character code of the character to be made active, and then lowercase the whole definition to apply this change:

begingroup
lccode`~=`!
lowercaseendgroupdeftestcatcode`!=active def~test
test
show!

Note that uppercase characters in the definition would be transformed to lowercase letters too.

share|improve this answer

edited 28 mins ago

answered 1 hour ago

siracusa

3,7341926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
  – Joseph Wright♦
  18 mins ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
  – Joseph Wright♦
  8 mins ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "85"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

 

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f454065%2fdefine-an-active-character-inside-a-definition%23new-answer', 'question_page');

);

Post as a guest

Name Email

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

active

oldest

votes

active

oldest

votes

up vote
3
down vote



accepted

The e-TeX reference defines scantokens as follows:

scantokens, when followed by a <general text>, decomposes the <balanced text> of the <general text> into the corresponding sequence of characters as if the <balanced text> were written unexpanded to a file; it then uses TeX's input mechanism to re-process these characters under the current catcode regime.

So the def! part of your macro is processed as if it were defined in an extra file that is then input into your main file. When TeX complains about File ended while scanning definition of !, it seems definitions of macros may not span across the end of a file and continue after that. I couldn't find a reference for that anywhere, but you can see the same behavior and error message if you replace scantokensdef! by an actual input were the input file includes only def!.

One way to get the desired result is by defining test in a regime which has the correct catcode already applied:

begingroup
catcode`!=active
gdeftestcatcode`!=active def!test
endgroup
test
show!

Instead of the gdef you could also save the current catcode of ! and restore it after the definition of test.

Another common trick is to change the lowercase code of an active character, here ~, to the character code of the character to be made active, and then lowercase the whole definition to apply this change:

begingroup
lccode`~=`!
lowercaseendgroupdeftestcatcode`!=active def~test
test
show!

Note that uppercase characters in the definition would be transformed to lowercase letters too.

share|improve this answer

edited 28 mins ago

answered 1 hour ago

siracusa

3,7341926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
  – Joseph Wright♦
  18 mins ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
  – Joseph Wright♦
  8 mins ago
  

  
  
  
  

add a comment | 

up vote
3
down vote



accepted

The e-TeX reference defines scantokens as follows:

scantokens, when followed by a <general text>, decomposes the <balanced text> of the <general text> into the corresponding sequence of characters as if the <balanced text> were written unexpanded to a file; it then uses TeX's input mechanism to re-process these characters under the current catcode regime.

So the def! part of your macro is processed as if it were defined in an extra file that is then input into your main file. When TeX complains about File ended while scanning definition of !, it seems definitions of macros may not span across the end of a file and continue after that. I couldn't find a reference for that anywhere, but you can see the same behavior and error message if you replace scantokensdef! by an actual input were the input file includes only def!.

One way to get the desired result is by defining test in a regime which has the correct catcode already applied:

begingroup
catcode`!=active
gdeftestcatcode`!=active def!test
endgroup
test
show!

Instead of the gdef you could also save the current catcode of ! and restore it after the definition of test.

Another common trick is to change the lowercase code of an active character, here ~, to the character code of the character to be made active, and then lowercase the whole definition to apply this change:

begingroup
lccode`~=`!
lowercaseendgroupdeftestcatcode`!=active def~test
test
show!

Note that uppercase characters in the definition would be transformed to lowercase letters too.

share|improve this answer

edited 28 mins ago

answered 1 hour ago

siracusa

3,7341926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
  – Joseph Wright♦
  18 mins ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
  – Joseph Wright♦
  8 mins ago
  

  
  
  
  

add a comment | 

up vote
3
down vote



accepted

up vote
3
down vote



accepted

The e-TeX reference defines scantokens as follows:

scantokens, when followed by a <general text>, decomposes the <balanced text> of the <general text> into the corresponding sequence of characters as if the <balanced text> were written unexpanded to a file; it then uses TeX's input mechanism to re-process these characters under the current catcode regime.

So the def! part of your macro is processed as if it were defined in an extra file that is then input into your main file. When TeX complains about File ended while scanning definition of !, it seems definitions of macros may not span across the end of a file and continue after that. I couldn't find a reference for that anywhere, but you can see the same behavior and error message if you replace scantokensdef! by an actual input were the input file includes only def!.

One way to get the desired result is by defining test in a regime which has the correct catcode already applied:

begingroup
catcode`!=active
gdeftestcatcode`!=active def!test
endgroup
test
show!

Instead of the gdef you could also save the current catcode of ! and restore it after the definition of test.

Another common trick is to change the lowercase code of an active character, here ~, to the character code of the character to be made active, and then lowercase the whole definition to apply this change:

begingroup
lccode`~=`!
lowercaseendgroupdeftestcatcode`!=active def~test
test
show!

Note that uppercase characters in the definition would be transformed to lowercase letters too.

share|improve this answer

edited 28 mins ago

answered 1 hour ago

siracusa

3,7341926

The e-TeX reference defines scantokens as follows:

scantokens, when followed by a <general text>, decomposes the <balanced text> of the <general text> into the corresponding sequence of characters as if the <balanced text> were written unexpanded to a file; it then uses TeX's input mechanism to re-process these characters under the current catcode regime.

So the def! part of your macro is processed as if it were defined in an extra file that is then input into your main file. When TeX complains about File ended while scanning definition of !, it seems definitions of macros may not span across the end of a file and continue after that. I couldn't find a reference for that anywhere, but you can see the same behavior and error message if you replace scantokensdef! by an actual input were the input file includes only def!.

One way to get the desired result is by defining test in a regime which has the correct catcode already applied:

begingroup
catcode`!=active
gdeftestcatcode`!=active def!test
endgroup
test
show!

Instead of the gdef you could also save the current catcode of ! and restore it after the definition of test.

Another common trick is to change the lowercase code of an active character, here ~, to the character code of the character to be made active, and then lowercase the whole definition to apply this change:

begingroup
lccode`~=`!
lowercaseendgroupdeftestcatcode`!=active def~test
test
show!

Note that uppercase characters in the definition would be transformed to lowercase letters too.

share|improve this answer

edited 28 mins ago

answered 1 hour ago

siracusa

3,7341926

share|improve this answer

share|improve this answer

edited 28 mins ago

edited 28 mins ago

edited 28 mins ago

answered 1 hour ago

siracusa

3,7341926

answered 1 hour ago

siracusa

3,7341926

answered 1 hour ago

siracusa

3,7341926

3,7341926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
  – Joseph Wright♦
  18 mins ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
  – Joseph Wright♦
  8 mins ago
  

  
  
  
  

add a comment | 

• 
  
  
  

  
  

  
  
  
  
  
  

  
  Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
  – Joseph Wright♦
  18 mins ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
  – Joseph Wright♦
  8 mins ago
  

  
  
  
  

Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
– Joseph Wright♦
18 mins ago

Normally one arranges that lowercase only applies to def~, e.g. deftestbegingrouplccode`~=`!lowercaseendgroupdef~test
– Joseph Wright♦
18 mins ago

On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
– Joseph Wright♦
8 mins ago

On scantokens, it is possible to avoid the issues with end-of-file but only by using an assignment. It's well-established that scantokens is a bit 'fragile': it needs careful handling.
– Joseph Wright♦
8 mins ago

add a comment | 

 

draft saved

draft discarded

 

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f454065%2fdefine-an-active-character-inside-a-definition%23new-answer', 'question_page');

);

Post as a guest

Name Email

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

Name Email

Name

Email

Name Email

Name

Email