AND logic gate in a neural network












1












$begingroup$


I'm creating a neural network like this one in Excel, which is https://www.youtube.com/watch?v=3993kRqejHc



N= X1·W1 + X2·W2 + X3·W3.



Is it compulsory to add the logic function that appears in column I? How is it called strictly that function in AI? Wouldn't the neural network work without it?






neural-networks







share|cite|improve this question









$endgroup$

















    1












    $begingroup$


    I'm creating a neural network like this one in Excel, which is https://www.youtube.com/watch?v=3993kRqejHc



    N= X1·W1 + X2·W2 + X3·W3.



    Is it compulsory to add the logic function that appears in column I? How is it called strictly that function in AI? Wouldn't the neural network work without it?






    neural-networks







    share|cite|improve this question









    $endgroup$















      1












      1








      1





      $begingroup$


      I'm creating a neural network like this one in Excel, which is https://www.youtube.com/watch?v=3993kRqejHc



      N= X1·W1 + X2·W2 + X3·W3.



      Is it compulsory to add the logic function that appears in column I? How is it called strictly that function in AI? Wouldn't the neural network work without it?






      neural-networks







      share|cite|improve this question









      $endgroup$




      I'm creating a neural network like this one in Excel, which is https://www.youtube.com/watch?v=3993kRqejHc



      N= X1·W1 + X2·W2 + X3·W3.



      Is it compulsory to add the logic function that appears in column I? How is it called strictly that function in AI? Wouldn't the neural network work without it?






      neural-networks




      neural-networks






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked Nov 30 '18 at 16:10









      galtorgaltor

      1085




      1085






















          1 Answer
          1






          active

          oldest

          votes


















          0












          $begingroup$

          That function is called an Activation Function, and it is actually what makes Neuronal Networks interesting.



          To give you an example, imagine you have a network with one input $x$ and one output $y$, if you ignore this function, the result is



          $$
          y = b + w x tag{1}
          $$



          where the parameters $b$ and $w$ are the numbers (weights) you need to find. The idea of training phase is to find the values of $a$ and $b$ that fit a bunch of training examples of the form ${x_i, y_i }$. So, you know what the output $y_1$ is when you feed the network an input $x_1$, also know what the output $y_2$ when you feed it $x_2$, ...



          Point here is that Eq. (1) is just a straight line, so no matter how many training examples you use. So you may ask yourself, isn't the problem of fitting a line already solved with ordinary least squares? And the answer is yes, it is, no need for neuronal networks at all!



          The obvious follow up question is then how to spice things up? And the answer will be, by introducing non-linearities



          $$
          y = f(b + w x) tag{2}
          $$



          There are several choices for $f$, this is a very simple one for binary classification



          $$
          f(x) = begin{cases} 0 & x < 0 \ 1 & {rm otherwise}end{cases}
          $$



          here is another one



          $$
          f_{rm sigmoid}(x) = frac{1}{1 + e^{-x}}
          $$



          and yet another one



          $$
          f_{rm ReLU}(x) = begin{cases} 0 & x < 0 \ x & {rm otherwise}end{cases}
          $$



          Each one has its merits, the last one has been recently used a lot for classifiers, but it kind of depends on the problem






          share|cite|improve this answer









          $endgroup$













          • $begingroup$
            If I have a hidden layer, does it have the same activation function as the output neuron?
            $endgroup$
            – galtor
            Nov 30 '18 at 22:57










          • $begingroup$
            @galtor not necessarily, you can have different activation functions. It just makes calculations easier
            $endgroup$
            – caverac
            Nov 30 '18 at 22:59










          • $begingroup$
            I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
            $endgroup$
            – galtor
            Dec 1 '18 at 22:01










          • $begingroup$
            @galtor What do you mean by 'identity activation function'? $f(x) = x$?
            $endgroup$
            – caverac
            Dec 1 '18 at 23:26










          • $begingroup$
            Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
            $endgroup$
            – galtor
            Dec 2 '18 at 6:19











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "69"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3020276%2fand-logic-gate-in-a-neural-network%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          0












          $begingroup$

          That function is called an Activation Function, and it is actually what makes Neuronal Networks interesting.



          To give you an example, imagine you have a network with one input $x$ and one output $y$, if you ignore this function, the result is



          $$
          y = b + w x tag{1}
          $$



          where the parameters $b$ and $w$ are the numbers (weights) you need to find. The idea of training phase is to find the values of $a$ and $b$ that fit a bunch of training examples of the form ${x_i, y_i }$. So, you know what the output $y_1$ is when you feed the network an input $x_1$, also know what the output $y_2$ when you feed it $x_2$, ...



          Point here is that Eq. (1) is just a straight line, so no matter how many training examples you use. So you may ask yourself, isn't the problem of fitting a line already solved with ordinary least squares? And the answer is yes, it is, no need for neuronal networks at all!



          The obvious follow up question is then how to spice things up? And the answer will be, by introducing non-linearities



          $$
          y = f(b + w x) tag{2}
          $$



          There are several choices for $f$, this is a very simple one for binary classification



          $$
          f(x) = begin{cases} 0 & x < 0 \ 1 & {rm otherwise}end{cases}
          $$



          here is another one



          $$
          f_{rm sigmoid}(x) = frac{1}{1 + e^{-x}}
          $$



          and yet another one



          $$
          f_{rm ReLU}(x) = begin{cases} 0 & x < 0 \ x & {rm otherwise}end{cases}
          $$



          Each one has its merits, the last one has been recently used a lot for classifiers, but it kind of depends on the problem






          share|cite|improve this answer









          $endgroup$













          • $begingroup$
            If I have a hidden layer, does it have the same activation function as the output neuron?
            $endgroup$
            – galtor
            Nov 30 '18 at 22:57










          • $begingroup$
            @galtor not necessarily, you can have different activation functions. It just makes calculations easier
            $endgroup$
            – caverac
            Nov 30 '18 at 22:59










          • $begingroup$
            I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
            $endgroup$
            – galtor
            Dec 1 '18 at 22:01










          • $begingroup$
            @galtor What do you mean by 'identity activation function'? $f(x) = x$?
            $endgroup$
            – caverac
            Dec 1 '18 at 23:26










          • $begingroup$
            Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
            $endgroup$
            – galtor
            Dec 2 '18 at 6:19
















          0












          $begingroup$

          That function is called an Activation Function, and it is actually what makes Neuronal Networks interesting.



          To give you an example, imagine you have a network with one input $x$ and one output $y$, if you ignore this function, the result is



          $$
          y = b + w x tag{1}
          $$



          where the parameters $b$ and $w$ are the numbers (weights) you need to find. The idea of training phase is to find the values of $a$ and $b$ that fit a bunch of training examples of the form ${x_i, y_i }$. So, you know what the output $y_1$ is when you feed the network an input $x_1$, also know what the output $y_2$ when you feed it $x_2$, ...



          Point here is that Eq. (1) is just a straight line, so no matter how many training examples you use. So you may ask yourself, isn't the problem of fitting a line already solved with ordinary least squares? And the answer is yes, it is, no need for neuronal networks at all!



          The obvious follow up question is then how to spice things up? And the answer will be, by introducing non-linearities



          $$
          y = f(b + w x) tag{2}
          $$



          There are several choices for $f$, this is a very simple one for binary classification



          $$
          f(x) = begin{cases} 0 & x < 0 \ 1 & {rm otherwise}end{cases}
          $$



          here is another one



          $$
          f_{rm sigmoid}(x) = frac{1}{1 + e^{-x}}
          $$



          and yet another one



          $$
          f_{rm ReLU}(x) = begin{cases} 0 & x < 0 \ x & {rm otherwise}end{cases}
          $$



          Each one has its merits, the last one has been recently used a lot for classifiers, but it kind of depends on the problem






          share|cite|improve this answer









          $endgroup$













          • $begingroup$
            If I have a hidden layer, does it have the same activation function as the output neuron?
            $endgroup$
            – galtor
            Nov 30 '18 at 22:57










          • $begingroup$
            @galtor not necessarily, you can have different activation functions. It just makes calculations easier
            $endgroup$
            – caverac
            Nov 30 '18 at 22:59










          • $begingroup$
            I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
            $endgroup$
            – galtor
            Dec 1 '18 at 22:01










          • $begingroup$
            @galtor What do you mean by 'identity activation function'? $f(x) = x$?
            $endgroup$
            – caverac
            Dec 1 '18 at 23:26










          • $begingroup$
            Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
            $endgroup$
            – galtor
            Dec 2 '18 at 6:19














          0












          0








          0





          $begingroup$

          That function is called an Activation Function, and it is actually what makes Neuronal Networks interesting.



          To give you an example, imagine you have a network with one input $x$ and one output $y$, if you ignore this function, the result is



          $$
          y = b + w x tag{1}
          $$



          where the parameters $b$ and $w$ are the numbers (weights) you need to find. The idea of training phase is to find the values of $a$ and $b$ that fit a bunch of training examples of the form ${x_i, y_i }$. So, you know what the output $y_1$ is when you feed the network an input $x_1$, also know what the output $y_2$ when you feed it $x_2$, ...



          Point here is that Eq. (1) is just a straight line, so no matter how many training examples you use. So you may ask yourself, isn't the problem of fitting a line already solved with ordinary least squares? And the answer is yes, it is, no need for neuronal networks at all!



          The obvious follow up question is then how to spice things up? And the answer will be, by introducing non-linearities



          $$
          y = f(b + w x) tag{2}
          $$



          There are several choices for $f$, this is a very simple one for binary classification



          $$
          f(x) = begin{cases} 0 & x < 0 \ 1 & {rm otherwise}end{cases}
          $$



          here is another one



          $$
          f_{rm sigmoid}(x) = frac{1}{1 + e^{-x}}
          $$



          and yet another one



          $$
          f_{rm ReLU}(x) = begin{cases} 0 & x < 0 \ x & {rm otherwise}end{cases}
          $$



          Each one has its merits, the last one has been recently used a lot for classifiers, but it kind of depends on the problem






          share|cite|improve this answer









          $endgroup$



          That function is called an Activation Function, and it is actually what makes Neuronal Networks interesting.



          To give you an example, imagine you have a network with one input $x$ and one output $y$, if you ignore this function, the result is



          $$
          y = b + w x tag{1}
          $$



          where the parameters $b$ and $w$ are the numbers (weights) you need to find. The idea of training phase is to find the values of $a$ and $b$ that fit a bunch of training examples of the form ${x_i, y_i }$. So, you know what the output $y_1$ is when you feed the network an input $x_1$, also know what the output $y_2$ when you feed it $x_2$, ...



          Point here is that Eq. (1) is just a straight line, so no matter how many training examples you use. So you may ask yourself, isn't the problem of fitting a line already solved with ordinary least squares? And the answer is yes, it is, no need for neuronal networks at all!



          The obvious follow up question is then how to spice things up? And the answer will be, by introducing non-linearities



          $$
          y = f(b + w x) tag{2}
          $$



          There are several choices for $f$, this is a very simple one for binary classification



          $$
          f(x) = begin{cases} 0 & x < 0 \ 1 & {rm otherwise}end{cases}
          $$



          here is another one



          $$
          f_{rm sigmoid}(x) = frac{1}{1 + e^{-x}}
          $$



          and yet another one



          $$
          f_{rm ReLU}(x) = begin{cases} 0 & x < 0 \ x & {rm otherwise}end{cases}
          $$



          Each one has its merits, the last one has been recently used a lot for classifiers, but it kind of depends on the problem







          share|cite|improve this answer












          share|cite|improve this answer



          share|cite|improve this answer










          answered Nov 30 '18 at 17:40









          caveraccaverac

          14.2k21130




          14.2k21130












          • $begingroup$
            If I have a hidden layer, does it have the same activation function as the output neuron?
            $endgroup$
            – galtor
            Nov 30 '18 at 22:57










          • $begingroup$
            @galtor not necessarily, you can have different activation functions. It just makes calculations easier
            $endgroup$
            – caverac
            Nov 30 '18 at 22:59










          • $begingroup$
            I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
            $endgroup$
            – galtor
            Dec 1 '18 at 22:01










          • $begingroup$
            @galtor What do you mean by 'identity activation function'? $f(x) = x$?
            $endgroup$
            – caverac
            Dec 1 '18 at 23:26










          • $begingroup$
            Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
            $endgroup$
            – galtor
            Dec 2 '18 at 6:19


















          • $begingroup$
            If I have a hidden layer, does it have the same activation function as the output neuron?
            $endgroup$
            – galtor
            Nov 30 '18 at 22:57










          • $begingroup$
            @galtor not necessarily, you can have different activation functions. It just makes calculations easier
            $endgroup$
            – caverac
            Nov 30 '18 at 22:59










          • $begingroup$
            I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
            $endgroup$
            – galtor
            Dec 1 '18 at 22:01










          • $begingroup$
            @galtor What do you mean by 'identity activation function'? $f(x) = x$?
            $endgroup$
            – caverac
            Dec 1 '18 at 23:26










          • $begingroup$
            Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
            $endgroup$
            – galtor
            Dec 2 '18 at 6:19
















          $begingroup$
          If I have a hidden layer, does it have the same activation function as the output neuron?
          $endgroup$
          – galtor
          Nov 30 '18 at 22:57




          $begingroup$
          If I have a hidden layer, does it have the same activation function as the output neuron?
          $endgroup$
          – galtor
          Nov 30 '18 at 22:57












          $begingroup$
          @galtor not necessarily, you can have different activation functions. It just makes calculations easier
          $endgroup$
          – caverac
          Nov 30 '18 at 22:59




          $begingroup$
          @galtor not necessarily, you can have different activation functions. It just makes calculations easier
          $endgroup$
          – caverac
          Nov 30 '18 at 22:59












          $begingroup$
          I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
          $endgroup$
          – galtor
          Dec 1 '18 at 22:01




          $begingroup$
          I'm trying to build a NAND logic gate with 3-2-1 structure. How can I know beforehand if it is possible to build it just using identity activation function?
          $endgroup$
          – galtor
          Dec 1 '18 at 22:01












          $begingroup$
          @galtor What do you mean by 'identity activation function'? $f(x) = x$?
          $endgroup$
          – caverac
          Dec 1 '18 at 23:26




          $begingroup$
          @galtor What do you mean by 'identity activation function'? $f(x) = x$?
          $endgroup$
          – caverac
          Dec 1 '18 at 23:26












          $begingroup$
          Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
          $endgroup$
          – galtor
          Dec 2 '18 at 6:19




          $begingroup$
          Yeap, exactly. Is there a general rule to check if my designed NN will be able to train a specific logic gate?
          $endgroup$
          – galtor
          Dec 2 '18 at 6:19


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Mathematics Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3020276%2fand-logic-gate-in-a-neural-network%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -