Integer types; createPerson function alternatives; Gas; Optimization; View v Pure functions; Non-iterable nature of mappings (by @jon_m)

gabba · September 21, 2020, 3:55pm

Hi guys
This topic is there to record the really interesting questions from @jon_m. And to keep them from getting lost in the solidity basics topics, as it can help a lot of us, I guess. I will therefore move his questions on this topic.

gabba · March 28, 2020, 6:18am

gabba · April 2, 2020, 11:12am

A post was merged into an existing topic: Solidity Basics

jon_m · April 2, 2020, 11:11am

@filip

I’ve really enjoyed this first section of the Solidity course - great material and great explanations!

Having finished this section on Solidity basics, I’m left with the following questions:

Question 1

I was also wondering about this. In the video I’m sure you say that uint (unsigned integer) can be used with both positive and negative integers. However, the following Wikipedia page suggests that unsigned only refers to positive numbers and zero.

This makes sense to me as we prefix a minus sign to represent a negative number value, but we don’t need to prefix a positive sign to represent a positive number value e.g.

uint positiveNumber = 5;

uint negativeNumber = -5;    // This threw an error when I tried it

Do you maybe mean that we can store as positive numbers what in reality are negative numbers, and then, as @marsrvr seems to suggest with decimal places, convert them into negative numbers with JavaScript in a front-end?

Question 2

In lesson 4 (Structs video) you showed us two alternatives for the createPerson function body:

struct Person {
   uint id;
   string name;
   uint age;
   uint height;
}

Person[] public people; 

// Alternative 1
function createPerson(string memory name, uint age, uint height) public {
   people.push(Person(people.length, name, age, height));
}

// Alternative 2
function createPerson(string memory name, uint age, uint height) public {
   Person memory newPerson;
   newPerson.id = people.length;
   newPerson.name = name;
   newPerson.age = age;
   newPerson.height = height;
   people.push(newPerson);
}

In the following videos you used Alternative 2 as the basis for developing the code examples for mappings and if...else control flow. I also attempted to write the equivalent using Alternative 1 - could you tell me if my alternative in the following code (2nd line in the function body) is correct?

struct Person {
   string name;
   uint age;
   uint height;
}

mapping(address => Person) private people; 

function createPerson(string memory name, uint age, uint height) public {
   address creator = msg.sender;
   people[creator] = Person(name, age, height);   // my alternative
}

Question 3

In JavaScript, the following would be valid alternatives to the if...else control flow demonstrated in lesson 8 (If & Else - Control Flow video). They also worked for me in Solidity, and I was wondering if you could confirm whether they are indeed valid alternatives:

// If & Else control flow demonstrated in the video
if (age >= 65) {
   newPerson.senior = true;
}
else {
   newPerson.senior = false;
}

// Alternative A - If & Else control flow without the curly brackets
if (age >= 65) newPerson.senior = true;
else newPerson.senior = false;

// Alternative B - using a ternary operator
age >= 65 ? newPerson.senior = true : newPerson.senior = false;

// Alternative C
newPerson.senior = age >= 65;

As Solidity seems to have a lot of similarities with JavaScript, and because Remix lets you know if your code is correct or contains errors, I’ve found that by using a trial-and-error approach I seem to be able to work out whether certain code variations I’ve tried are valid or not (for example my control flow alternatives above). Is this a good way to experiment with Solidity if you already have a good grounding in JavaScript? It does avoid having to look everything up in documentation. Or is this trial-and-error approach (until the compiler gives you a green tick) dangerous to rely on, for reasons such as getting into bad practice? Should I always be using documentation instead? As there does seem to be a lot of Solidity syntax which is exactly the same as JavaScript, is there any kind of reliable summary available anywhere, which gives you a good quick reference as to what is the same and what is different? I would have thought a reference of this kind would certainly speed up learning Solidity for people who already know a lot of JavaScript…

Question 4

In the quiz on mappings:

3. Is it possible to find all values entered into a mapping in Solidity? Answer: No

I put Yes because I thought that if it was a mapping which wasn’t based on user addresses, then if all the mapping’s key values were known and the getter function allowed the user to input these key values one at a time, then essentially you could retrieve all of the values mapped to those key values — not altogether, but one by one. Is that a valid interpretation of the question?

Is the answer No because the question refers to the type of mapping that we have in our example, where any user executing the contract would only be able to retrieve the value in the mapping associated with their own address? Or does it refer to the fact that only individual values can be retrieved each time the getter function is called, and never all values altogether? If not, what is the reasoning behind the correct answer?

gabba · April 2, 2020, 11:11am

Hi @jon_m

Q1:

Playing with cast is not a good solution (it s a bit dangerous ) and solidity allow you to use regular integer, in this example you can see that you can use negative number. You can also see the effect when you are not respecting the type limit (i used int8 and uint8 as the limit is more human friendly )

pragma solidity >=0.4.22 <0.7.0;

contract Test {

    
    uint8 test1 = 0;
    int8 test2 = 0;
    
    // Uint can be 0 to 255 otherwhise we overflow or underflow
    function getUintLimit() public view returns(uint8, uint8){
         // The minimum value is zero if we overlap we go back to the max number
        uint8 underFlow = test1 - 1;
         // The maximum value is 255 if we overlap we go back to the min number
        uint8 overflow = (uint8)(test1 + 256); // I have to cast here because the compiler detect an overflow
        return (underFlow, overflow);
    }

    // Limit -127 and 127
    function getIntLimit() public view returns(int8, int8){
        int8 LimitDownOk = test2 - 127; // OK
        //int8 underFlow = (int8)(test2 - 128); // throw an error
        int8 LimitUpOk = test2 + 127; // OK
        //int8 overFlow = (int8)(test2 + 128); // throw an error   
        return (LimitDownOk, LimitUpOk);
    }
    
}

Q2:
Your alternative is correct, but keep in mind that an array and a map are not used for the same thing.
If i asked you how many peoples are stored in your contract, or what is the age of “Bob” your alternative will not works. But to access a Person information your alternative is faster.

Q3:
The problem is clarity you can use all of them but if someone else is trying to read you code the first is the best, i use ternary often for simple functions. The second and the fourth are really not human readable IMO
Using solidity compiler is faster, but the documentation always have better explanation. I don’t know a good website which compare solidity and javascript syntax if you find one please share it with us on a new topic

Q4:
You can have a look at my explanation in this post:

This question is a bit tricky it depends a lot on your interpretation, i think the point here is, is it possible to iter on each elements of a mapping.

jon_m · April 2, 2020, 11:11am

Hey! Thank you so much for your detailed response

I’ve had a really good look and think about everything you’ve explained and outlined, and I’d just like to confirm a few things and ask a few more questions…

Q1

Thanks for introducing the concept of cast. I didn’t know what that was, but I’ve looked it up and now I do

So, basically, we are saying that
(i) uint only allows zero and positive integers up to a max based on the binary places included in the type e.g. uint4 (0 to 15) , uint8 (0 to 255) , uint16 (0 to 65535)
(ii) int allows positive and negative integers from a min of minus(-) half the max based on the binary places included in the type, to a max of plus(+) half the max based on the binary places, and including zero (as the median of this range).
e.g. int4 (-7 to 7) , int8 (-127 to 127) , int16 (-32767 to 32767)
Is that correct?

I understand the concept of underflow and overflow that you have demonstrated in your example code. Are you trying to demonstrate that this happens with uint but not with int (hence the errors thrown with int when trying to store integers above/below the max/min)? I have proved to myself that the underflow does indeed happen with uint and the following line of code stores the max (255 using uint8):

However, I don’t really understand the following line of code from your example, and why you refer to this as casting. If test1 is already defined in a state variable as type unit8, and our new variable overflow is also defined as type unit8, why do we need to add the additional (unit8) to the assigned expression, and why the parentheses? Also, even with this additional (unit8), my Remix compiler still throws an error here…

Why does the compiler perform the underflow with uint (returning to the max) but not the overflow (i.e. why doesn’t it return to the min)?

Q2

I take your point about the differences between arrays and mappings, but @filip had already provided two alternatives for the array approach in the Structs video:

jon_m:

struct Person {
   uint id;
   string name;
   uint age;
   uint height;
}

Person[] public people; 

// Alternative 1
function createPerson(string memory name, uint age, uint height) public {
   people.push(Person(people.length, name, age, height));
}

// Alternative 2
function createPerson(string memory name, uint age, uint height) public {
   Person memory newPerson;
   newPerson.id = people.length;
   newPerson.name = name;
   newPerson.age = age;
   newPerson.height = height;
   people.push(newPerson);
}

My alternative was purely meant as an alternative to @filip’s code in the Mappings video, where he only provides one version, which is based on Alternative 2 for the array in the Structs video. Despite the differences you mention between using arrays and mappings, I think you are confirming that my alternative for the mapping is correct, aren’t you? Below, I’ve included @filip’s version next to mine, which will hopefully make it clearer to see what I was hoping to achieve…

struct Person {
   string name;
   uint age;
   uint height;
}

mapping(address => Person) private people;

// Filip's version (based on the previous Alternative 2)
function createPerson(string memory name, uint age, uint height) public {
   address creator = msg.sender;   

   Person memory newPerson;
   newPerson.name = name;
   newPerson.age = age;
   newPerson.height = height;

   people[creator] = newPerson;
}

// My alternative (based on the previous Alternative 1)
function createPerson(string memory name, uint age, uint height) public {
   address creator = msg.sender;
   people[creator] = Person(name, age, height);
}

Q3

Thanks for confirming my alternatives are correct. I take your point about clarity, but I guess that is something purely subjective, and people can have different opinions about what is clearer and what isn’t. Would there be differences in the cost of gas between the alternatives, making this a possible factor in deciding which to use?

Will do!

Q4

gabba · April 2, 2020, 11:11am

Hi @jon_m

Q1:
Yeah it’s exact you get it
Regarding my example i just rush a bit righting it. You can have overflow or underflow in any type, none are safe. I shouldn’t had use an other variable you can try this code it ll bug in both case:

    // Uint can be 0 to 255 otherwhise we overflow or underflow
    function getUintLimit() public view returns(uint8, uint8){
         // The minimum value is zero if we overlap we go back to the max number
        uint8 underFlow = 0;
        underFlow -= 1;
         // The maximum value is 255 if we overlap we go back to the min number
        uint8 overflow = 0;
        overflow = (uint8)(overflow + 256); // I have to cast here because the compiler detect an overflow
        return (underFlow, overflow);
    }

    // Limit -127 and 127
    function getIntLimit() public view returns(int8, int8){
        int8 LimitDownOk = 0;
        int16 testU = 128;
        int16 testD = 129;
        LimitDownOk  = (int8)(LimitDownOk - testD);
      
        int8 LimitUpOk = 0; 
        LimitUpOk  = (int8)(LimitUpOk + testU);
 
        return (LimitDownOk, LimitUpOk);
    }

I had to cast or use a variable of an other type, because the compiler is detecting that 256 couldn’t be an uint8 same for a number up to 128 it can’t be an int8. So it’s just a trick to be able to show a bad pratice

Q2:
For this use case the result is the same, actually faster with a mapping.Your alternative works and it’s more compact.

Q3:
It could be, actually the gas cost is base on how many opcodes are used. For an if else statement it’s gonna be the same opcodes because this is a simple operation i guess.
But you can try to compile the same contract with a different statement and look at the opcode generated.
If you click on the Debug button in remix you can follow the cost of each operation in gas.

jon_m · April 2, 2020, 11:11am

I’ve finally worked this through…

Q1

Having experimented some more with this, I can now see that sometimes the compiler blocks an underflow or overflow by throwing an error, but that by doing some casting or adding additional variables of a different type, we can “force” the underflow and overflow to occur with both uint and int , and that this is…

In other words, we should always keep within the restrictions in terms of the range of integers available with a given integer type (signed or unsigned), because the compiler will not always prevent an overflow (a loop from the max limit, back to the min limit) or an underflow (a loop from the min limit back to the max limit).

Is that correct?

By the way, the view in your example functions throws the following warning in my Remix compiler:

Warning: Function state mutability can be restricted to pure

The functions still work when deployed with view (ignoring the warnings) but the warnings disappear when I change view for pure . What exactly is pure and why does the compiler recommend it be used instead of view ?

Also, having experimented a bit, I’ve also found that the following statements in your code:

overflow = (uint8)(overflow + 256);
LimitDownOk = (int8)(LimitDownOk - testD);
LimitUpOk = (int8)(LimitUpOk + testU);

…can be reduced to…

overflow += uint8(256);
LimitDownOk -= int8(testD);
LimitUpOk += int8(testU);

What do you think?

I’ve also spotted that, with int types, the negative integer limit is always 1 more than the positive integer limit, to compensate for the fact that zero is included as the median of the full range i.e.

int4 (-8 to 7) , int8 (-128 to 127) , int16 (-32768 to 32767)

…and not…

Q2 resolved

Q3

Thanks — I’ve been able to do this in the Debugger area, and I’ve found some differences in the gas cost between the alternative control flow statements we’ve considered.
When I total the individual gas costs for all the operations related to each alternative control flow statement, the differences between these totals are also equal to the differences between the total execution costs shown in the transaction details.

// Ternary operator (highest gas cost = 80)
// total execution cost of tx = 63895 gas   
age >= 65 ? newPerson.senior = true : newPerson.senior = false;

// if...else statement (gas cost = 67)
// total execution cost of tx = 63882 gas
// -13 gas difference
if (age >= 65) {
   newPerson.senior = true;
}
else {
   newPerson.senior = false;
}

// Boolean expression assigned directly (lowest gas cost = 52)
// total execution cost of tx = 63867 gas
// -28 gas difference
newPerson.senior = age >= 65;

gabba · April 2, 2020, 11:11am

Yes the compiler will not prevent your code to underflow or overflow, if you take the ethereum security course you ll learn a lot about it.

A view function doesn’t allow you to modify the storage state but you can still read the storage. A pure function doesn’t allow you to modify and read the storage, only local variable will be used.
Strongly typing your function helps to avoid making mistake.

Yes this is better

overflow += uint8(256);
LimitDownOk -= int8(testD);
LimitUpOk += int8(testU);

But for an example i try to keep it simple to show the point

int4 (-8 to 7) , int8 (-128 to 127) , int16 (-32768 to 32767)

Correct


// Ternary operator (highest gas cost = 80)
// total execution cost of tx = 63895 gas   

// if...else statement (gas cost = 67)
// total execution cost of tx = 63882 gas
// -13 gas difference

// Boolean expression assigned directly (lowest gas cost = 52)
// total execution cost of tx = 63867 gas
// -28 gas difference

Woo really nice, i’m really surprise that ternary cost more gas thank to run this tests.
Btw did you check the Enable optimization box before compiling ?

Thank a lot for all your observations @jon_m you did a lot of great research.
Btw if you have time (i know you already spend a lot of time writing this post) can you create a new topic on the forum, and share with us all your observations. I m sure it ll help a lot of people and this topic is more about the basic, i think you got far further than the basic

jon_m · April 7, 2020, 2:01am

Copied from our private conversation where we had continued this discussion:

My tests were without this enabled. I’ve now re-run them with it enabled, and I’ll post my findings in the new topic when it’s set up. What exactly does Enable optimization do? What does it optimize? I couldn’t find anything about this in the documentation…

I’m still not sure of all the terminology:
Does storage state refer to values stored in state variables — variables defined at the beginning of the contract and outside of the functions?
Are local variables ones that are defined within functions (like in the overflow/underflow test functions we have used)?
If the answer is yes in both cases, is the compiler suggesting we change view to pure because our test functions only need to read variables that are defined and modified within these same functions?

What do you mean by strongly typing ? Does it mean making a function pure instead of just view when only local variables (and not state variables) need to be accessed?

gabba · April 7, 2020, 1:54pm

yeah i ll try to do that today

The optimization will make the compiler able to find duplicated op code or/and remove useless code, their is a good example here.

Yes as you can see here:

    int8 test2 = 0;

    function getIntLimit() public pure returns(int8, int8){
        int8 LimitDownOk = 0;
        LimitDownOk -= 127; // Pure

        int8 notPure = test2 + 127;

        return (LimitDownOk, notPure);
    }

TypeError: Function declared as pure, but this expression (potentially) reads from the environment or state and thus requires “view”.
int8 notPure = test2 + 127;
^—^

Yes.
I mean it will helps you to avoid human mistake for example you are designing your contract’s function and you create a function which is just applying a mathematical operation and not modifying the state of your contract.

This function shouldn’t modify or read the state, when an other developer (or yourself) add code to this function if they are not following your function declaration a warning or an error will be throw.

It will help other developers to know what is the real purpose of this function for example:

    //function isNbrOdd(uint8 isOdd) public view returns(bool){
    function isNbrOdd(uint8 isOdd) public pure returns(bool){
        if (isOdd % 2 == 0)
            return true;
        return false;
    }

Is pure but you can also compile it as a view function (with a warning).

But if someone try to read the state in this function it ll throw an error:

   bool test = true;
    function isNbrOdd(uint8 isOdd) public pure returns(bool){
        if (isOdd % 2 == 0)
            return test;
        return false;
    }

You will only be able to compile it as a view function.

When you have a 200 lines of code function to review if you had use the right type when declaring it this kind of issue will not happened.
But this function can be declare as a view function if you decide that in a future implementation it ll be possible to read the state from it (and ignore the warning).

jon_m · May 16, 2020, 8:21pm

@gabba @firepol

Results with optimization ON

There is no change in the order from highest → lowest gas cost. In fact the difference between the costs is actually greater; so, relatively speaking, the ternary operator has an even higher gas cost than the other two control flow options:

Ternary operator:
Still the highest gas cost = 75 (only 5 less than with optimization OFF)
age >= 65 ? newPerson.senior = true : newPerson.senior = false;

if…else statement
Gas cost = 54 (13 less than with optimization OFF)
-21 gas difference compared to ternary operator (additional -8 than with optimization OFF)

if (age >= 65) {
   newPerson.senior = true;
}
else {
   newPerson.senior = false;
}

Boolean expression assigned directly:
Still the lowest gas cost = 39 (also 13 less than with optimization OFF)
-36 gas difference compared to ternary operator (also an additional -8 than with optimization OFF)
newPerson.senior = age >= 65;

dan-i · October 26, 2020, 4:08pm