{"id":93,"date":"2019-10-07T16:45:20","date_gmt":"2019-10-07T15:45:20","guid":{"rendered":"https:\/\/computingskillset.com\/?p=93"},"modified":"2019-10-13T13:01:34","modified_gmt":"2019-10-13T12:01:34","slug":"highly-instructive-examples-for-the-newton-raphson-method","status":"publish","type":"post","link":"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/","title":{"rendered":"11 Highly Instructive Examples for the Newton Raphson Method"},"content":{"rendered":"\n<p>Newton&#8217;s method for numerically finding roots of an equation is most easily understood by example. At least, I learn more easily from examples. So, perhaps you do, too. <\/p>\n\n\n\n<p><strong>In this article I&#8217;ve collected a couple of highly instructive examples for the Newton-Raphson method and for what it does. You&#8217;ll see it work nicely and fail spectacularly. I show you the equations and pictures you need to understand what happens, and I give you a piece of python code so that you can try all that yourself.<\/strong><\/p>\n\n\n\n<p>Before we dive into the examples, let me mention that I have written a complete introduction to Newton&#8217;s method itself, here on ComputingSkillSet.com, in my article <a href=\"https:\/\/computingskillset.com\/solving-equations\/the-newton-raphson-method-explained-details-pictures-python-code\/\">Newton\u2019s Method Explained: Details, Pictures, Python Code<\/a>. If at any step in one of the examples you feel like you could benefit from looking at an explanation of this particular step, just jump over there and come back afterwards.<\/p>\n\n\n\n<p>And one more thing: Newton&#8217;s method for finding roots of a function is also called the Newton-Raphson method, so I&#8217;ll use these two names interchangeably.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_65 counter-hierarchy ez-toc-counter ez-toc-white ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title \" >Here you&#039;ll learn more about:<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-1-calculating-square-roots-of-positive-numbers-with-newtons-method\" title=\"Example 1: calculating square roots of positive numbers with Newton&#8217;s method\">Example 1: calculating square roots of positive numbers with Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-2-calculating-cubic-roots-of-positive-numbers-with-newtons-method\" title=\"Example 2: calculating cubic roots of positive numbers with Newton&#8217;s method\">Example 2: calculating cubic roots of positive numbers with Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-3-calculating-any-roots-of-positive-numbers-with-newtons-method\" title=\"Example 3: calculating any roots of positive numbers with Newton&#8217;s method\">Example 3: calculating any roots of positive numbers with Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-4-newtons-method-fails-when-there-is-no-root\" title=\"Example 4: Newton&#8217;s method fails when there is no root \">Example 4: Newton&#8217;s method fails when there is no root <\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-5-newtons-method-running-away-into-an-asymptotic-region-of-the-function\" title=\"Example 5: Newton&#8217;s method running away into an asymptotic region of the  function\">Example 5: Newton&#8217;s method running away into an asymptotic region of the  function<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-6-newtons-method-oscillating-between-two-regions-forever\" title=\"Example 6: Newton&#8217;s method oscillating between two regions forever\">Example 6: Newton&#8217;s method oscillating between two regions forever<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-7-newtons-method-fails-for-roots-rising-slower-than-a-square-root\" title=\"Example 7: Newton&#8217;s method fails for roots rising slower than a square root\">Example 7: Newton&#8217;s method fails for roots rising slower than a square root<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-8-newtons-method-for-the-arctangent-function\" title=\"Example 8: Newton&#8217;s method for the arctangent function\">Example 8: Newton&#8217;s method for the arctangent function<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-9-a-couple-of-roots-to-choose-from-for-newtons-method\" title=\"Example 9: A couple of roots to choose from for Newton&#8217;s method\">Example 9: A couple of roots to choose from for Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-10-fractals-generated-with-newtons-method\" title=\"Example 10: Fractals generated with Newton&#8217;s method\">Example 10: Fractals generated with Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#example-11-finding-roots-with-higher-multiplicity-in-polynomials-with-newtons-method\" title=\"Example 11: Finding roots with higher multiplicity in polynomials with Newton&#8217;s method\">Example 11: Finding roots with higher multiplicity in polynomials with Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#python-code-to-generate-the-solutions-and-figures-for-these-examples-for-newtons-method\" title=\"Python code to generate the solutions and figures for these examples for Newton&#8217;s method\">Python code to generate the solutions and figures for these examples for Newton&#8217;s method<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/computingskillset.com\/solving-equations\/highly-instructive-examples-for-the-newton-raphson-method\/#further-information-about-the-newton-raphson-method\" title=\"Further information about the Newton-Raphson method\">Further information about the Newton-Raphson method<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-1-calculating-square-roots-of-positive-numbers-with-newtons-method\"><\/span>Example 1: calculating square roots of positive numbers with Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Newton&#8217;s method can be used to calculate the square root of a positive real number. Sounds interesting, doesn&#8217;t it? But how does this computational algorithm help to compute square roots? Here is how:<\/p>\n\n\n\n<p>Let&#8217;s say that we are interested in the square root of 2. Then, we look at the function<\/p>\n\n\n\n\\[f(x)=x^2-2\\]\n\n\n\n<p>Why this function? Because the way Newton&#8217;s method works is that it can help us find zeros of functions, if we already have a faint idea as to where such a zero might be. In order to make use of this strategy, the function above is designed such that it has a zero at, yes indeed, the square root of two:<\/p>\n\n\n\n<p>\\[f(x)=0\\quad\\rightarrow\\quad x^2-2=0\\quad\\rightarrow\\quad x^2=2\\]<\/p>\n\n\n\n<p>This means that if we use Newton&#8217;s method to find the zero (root) of this function, it will give us a numerical value of the square root of two. At this point, it is already clear, by the way, what to do, if you&#8217;d want the square root of seven instead: You&#8217;d use \\(f(x)=x^2-7\\). But because we want to see a number come out that many people recognize, let&#8217;s stick to the square root of two.<\/p>\n\n\n\n<p>Here is an overview of what this function and its derivative look like:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"557\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-squarerootoftwo.jpg\" alt=\"Searching for the square root of two with Newton's Method: Overview\" class=\"wp-image-117\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-squarerootoftwo.jpg 557w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-squarerootoftwo-300x238.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-squarerootoftwo-20x16.jpg 20w\" sizes=\"(max-width: 557px) 100vw, 557px\" \/><\/figure><\/div>\n\n\n\n<p>Newton&#8217;s method tells you to <a href=\"https:\/\/computingskillset.com\/solving-equations\/how-to-find-the-initial-guess-in-newtons-method\/\">start with an initial guess<\/a>, \\(x_0\\), and then iterate the following line until you are either satisfied with the result or the algorithm fails:<\/p>\n\n\n\n<p>\\[x_{i+1}= x_i &#8211; \\frac{f(x_i)}{f'(x_i)} \\]<\/p>\n\n\n\n<p>For our concrete example function, this becomes<\/p>\n\n\n\n<p>\\[x_{i+1}= x_i &#8211; \\frac{x_i^2-2}{2x_i} \\]<\/p>\n\n\n\n<p>and that is fairly easy to do, even by hand, if we&#8217;d like.<\/p>\n\n\n\n<p>Ok, first we pick an initial guess. We know (playing naive on purpose here) that the square root of two lies somewhere between the numbers 1 and 2, so let&#8217;s pick \\(x_0=1.5\\), that&#8217;s right in the middle. Then we get <\/p>\n\n\n\n<p>\\[x_1= 1.5 &#8211; \\frac{1.5^2-2}{3}= 1.5 &#8211; \\frac{2.25-2}{3}= 1.5 &#8211; \\frac{0.25}{3}=1.5 -0.0833333= 1.4166666 \\]<\/p>\n\n\n\n<p>So this is the first step. One thing that we need to keep an eye on is the relative difference in the \\(x_i\\) from one step to the next, namely:<\/p>\n\n\n\n<p>\\[\\frac{|x_{i+1}- x_i|}{|x_i|}= \\frac{|f(x_i)|}{|f'(x_i)x_i|} \\]<\/p>\n\n\n\n<p>For our particular example and step, we have<\/p>\n\n\n\n<p>\\[\\frac{|f(x_0)|}{|f'(x_0)x_0|}=0.0555555 \\]<\/p>\n\n\n\n<p>Ok, next step. This time, doing the calculation without a calculator isn&#8217;t so much fun anymore, but I&#8217;ll still note the numbers as if we&#8217;d do it that way:<\/p>\n\n\n\n<p>\\[x_2= 1.4166666 &#8211; \\frac{1.4166666^2-2}{2.8333333}= 1.416666666 &#8211; 0.00245098= 1.414215686 \\]<\/p>\n\n\n\n<p>To those of you who know the answer by heart, this should already look pretty good. To keep our score, let&#8217;s check the relative difference. It is<\/p>\n\n\n\n<p>\\[\\frac{|f(x_1)|}{|f'(x_1)x_1|}=0.0017301 \\]<\/p>\n\n\n\n<p>On to the next step. We get<\/p>\n\n\n\n<p>\\[x_3= 1.414215686275 &#8211; \\frac{1.414215686275^2-2}{2.828427137255}= 1.414215686275 &#8211; 0.000002123900= 1.414213562375 \\]<\/p>\n\n\n\n<p>You have certainly noticed that I keep adding more and more digits to those numbers, and that&#8217;s because it is necessary. The result is getting closer to the actual root of our equation really quickly now. Let&#8217;s note the relative difference for this step:<\/p>\n\n\n\n<p>\\[\\frac{|f(x_2)|}{|f'(x_2)x_2|}=0.0000015 \\]<\/p>\n\n\n\n<p>So, because the next step should (and will) be smaller than this one, we can estimate that our result is already precise to 5 digits or more. One more step, ok? Here we go:<\/p>\n\n\n\n<p>\\[x_3= 1.414213562375 &#8211; \\frac{1.414213562375^2-2}{2.82842712475}= 1.414213562375 &#8211; 0.000000000002= 1.414213562373 \\]<\/p>\n\n\n\n<p>Just a tiny change. In fact, the relative difference is \\(0.000000000001\\) and we are done. This worked really well and fairly quickly, because we were close to the root and the function is well-behaved. To illustrate this, here is a plot of the tangents that were constructed to go these steps. All of them are very close and it is hard to see anything:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"571\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-squarerootoftwo.jpg\" alt=\"Searching for the square root of two with Newton's Method: The search\" class=\"wp-image-120\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-squarerootoftwo.jpg 571w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-squarerootoftwo-300x232.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-squarerootoftwo-20x15.jpg 20w\" sizes=\"(max-width: 571px) 100vw, 571px\" \/><\/figure>\n\n\n\n<p>The function is plotted in blue and hard to see underneath the red-dashed tangents. The red stars mark those points on the curve, where the steps are. The green Xs mark the same points on the x-axis, and green dashed lines lead up to the respective stars.<\/p>\n\n\n\n<p>While for this particular choice of the initial guess, the convergence is fast, it is different for other initial guesses. To get an idea, how different it is, we can plot the number of iterations steps needed in order to reach a certain level of relative precision, in our case \\(10^{-10}\\). The result is the following figure:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"570\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-squareroot.jpg\" alt=\"Searching for the square root of two with Newton's Method: Dependence on the initial guess\" class=\"wp-image-121\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-squareroot.jpg 570w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-squareroot-300x233.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-squareroot-20x16.jpg 20w\" sizes=\"(max-width: 570px) 100vw, 570px\" \/><\/figure><\/div>\n\n\n\n<p>These red dots indicate areas of fast convergence, where they are low, and slow convergence, where they are high. In general, a small slope of the function in question at a certain initial guess leads to slower convergence. In our example, the origin is particularly unsuitable, because there the slope vanishes and so the Newton algorithm fails.<\/p>\n\n\n\n<p>In summary, we have seen that Newton&#8217;s method can be used for getting numerical values for square roots of positive real numbers in a straightforward manner.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-2-calculating-cubic-roots-of-positive-numbers-with-newtons-method\"><\/span>Example 2: calculating cubic roots of positive numbers with Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>This next example is similar to the first, but would be a little more annoying to do by hand. We&#8217;ll use the Newton-Raphson method to compute the cubic root of the number 2. This is a number that isn&#8217;t as familiar as the square root of two, but it is easy enough to check on a pocket calculator that can do cubic roots. So here we go.<\/p>\n\n\n\n<p>The function we define for this purpose is (can you guess it after having read the first example?):<\/p>\n\n\n\n\\[f(x)=x^3-2\\]\n\n\n\n<p>This function has the property of having a (single) root at \\(x=\\sqrt[3]{2}\\), which is what we want to know and what Newton&#8217;s method will help us find. The function and its derivative look like this:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"566\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-cuberootoftwo.jpg\" alt=\"Searching for the cubic root of two with Newton's Method: Overview\" class=\"wp-image-125\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-cuberootoftwo.jpg 566w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-cuberootoftwo-300x234.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-cuberootoftwo-20x16.jpg 20w\" sizes=\"(max-width: 566px) 100vw, 566px\" \/><\/figure><\/div>\n\n\n\n<p>To start the search, let&#8217;s pick an initial guess, say \\(x_0 = 1.8\\). With the general formula<\/p>\n\n\n\n<p>\\[x_{i+1}= x_i &#8211; \\frac{f(x_i)}{f'(x_i)} \\]<\/p>\n\n\n\n<p>we find for our cubic-root generating function<\/p>\n\n\n\n<p>\\[x_{i+1}= x_i &#8211; \\frac{x_i^3-2}{3x_i^2} \\]<\/p>\n\n\n\n<p>Applying this to \\(x_0 = 1.8\\), we arrive at \\(x_1 = 1.405761316872\\) and a relative difference of \\(0.219021490626\\). Repeating the procedure (for more details, see example 1 above), we get  \\(x_2 = 1.274527978340\\) with a relative difference of \\(0.093353926415\\),  \\(x_3 = 1.260087815373\\) with a relative difference of \\(0.011329812458\\),   \\(x_4 = 1.259921071964\\) with a relative difference of \\(0.000132326816\\),   \\(x_5 = 1.259921049895\\) with a relative difference of \\(0.000000017517\\),  and \\(x_6 = 1.259921049895\\) with a relative difference of \\(0.000000000000\\). Yes, that&#8217;s right &#8211; the last step didn&#8217;t change anything in the digits shown here. So that is our result:<\/p>\n\n\n\n<p>\\[\\sqrt[3]{2}=1.259921049895\\]<\/p>\n\n\n\n<p>In a picture, these steps can again be visualized by the tangents that correspond to them:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"558\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-cuberootoftwo.jpg\" alt=\"Searching for the cubic root of two with Newton's Method: Search\" class=\"wp-image-126\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-cuberootoftwo.jpg 558w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-cuberootoftwo-300x238.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-cuberootoftwo-20x16.jpg 20w\" sizes=\"(max-width: 558px) 100vw, 558px\" \/><\/figure><\/div>\n\n\n\n<p>In this case, at least the first tow steps and tangents are clearly distinguishable from the rest, before the plot becomes crowded near the actual value of the root. Again, we can ask about the dependence of the number of necessary steps on the initial guess. Testing this and plotting this number across the entire range plotted in the overview, we get:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"566\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-cuberootoftwo.jpg\" alt=\"Searching for the cubic root of two with Newton's Method: Dependence on the initial guess\" class=\"wp-image-127\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-cuberootoftwo.jpg 566w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-cuberootoftwo-300x234.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-cuberootoftwo-20x16.jpg 20w\" sizes=\"(max-width: 566px) 100vw, 566px\" \/><\/figure><\/div>\n\n\n\n<p>The low regions in the red-dot-curve provide a good solution after a low number of iteration steps, while the high regions take longer. The point to avoid is once again the origin, where the slope of our function vanishes and the algorithm of the Newton-Raphson method stops.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-3-calculating-any-roots-of-positive-numbers-with-newtons-method\"><\/span>Example 3: calculating any roots of positive numbers with Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>This example also deals with calculating roots, but I&#8217;d like to keep this brief. You already got the idea from the first two with the concrete numbers and the illustrations. In this example, I want to just write down the general formula for computing the \\(n^{th}\\) root of the number \\(a\\). <\/p>\n\n\n\n<p>In order to find a number that is the \\(n^{th}\\) root (\\(n&gt;1\\)) of the positive real number \\(a\\), which function do we need to use in Newton&#8217;s method? We use <\/p>\n\n\n\n\\[f(x)=x^n-a\\]\n\n\n\n<p>because this is zero, where \\(x=\\sqrt[n]{a}\\). Now, recall the Newton-Raphson iteration step, which is<\/p>\n\n\n\n<p>\\[x_{i+1}= x_i &#8211; \\frac{f(x_i)}{f'(x_i)} \\]<\/p>\n\n\n\n<p>and substitute for \\(f\\) and \\(f&#8217;\\) accoringly. That brings us to<\/p>\n\n\n\n<p>\\[x_{i+1}= x_i &#8211; \\frac{x_i^n-a}{n x_i^{n-1}} \\]<\/p>\n\n\n\n<p>In much the same way, we can construct the iteration step explicitly in terms of the function we want to investigate. This works, provided that we can write down a functional form for the function and its derivative.<\/p>\n\n\n\n<p>In the next couple examples, we&#8217;ll see and experience the various ways, in which Newton&#8217;s method can and will fail.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-4-newtons-method-fails-when-there-is-no-root\"><\/span>Example 4: Newton&#8217;s method fails when there is no root <span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In my article <a href=\"https:\/\/computingskillset.com\/solving-equations\/the-newton-raphson-method-explained-details-pictures-python-code\/\">Newton\u2019s Method Explained: Details, Pictures, Python Code<\/a>, I mention a number of cases where Newton&#8217;s method can or will fail. In the following, I present some examples for exactly those cases. The first of these is that the function, for which we try to find a root, doesn&#8217;t actually have any.<\/p>\n\n\n\n<p>Now you might say that this is an artificial situation, because we usually know whether the functions we investigate have roots or not. While this is true for something like \\(f(x)=x^2\\), it can indeed be more complicated in general. <\/p>\n\n\n\n<p>Think about the following situation: you tell me about some interesting phenomenon. We figure out an equation together that&#8217;s supposed to describe this phenomenon. But our equation isn&#8217;t algebraic to begin with, so we have no idea what the solution looks like, we just know that it should be a function of a single variable, whose most important part is its zero (root).  So in the first step, we need to calculate the function for which we&#8217;d like to use Newton&#8217;s method. And then it could well happen that there is no root for this function. Ok, enough with the mind experiments, let&#8217;s look at an example.<\/p>\n\n\n\n<p>Above, we have used the function <\/p>\n\n\n\n\\[f(x)=x^2-2\\]\n\n\n\n<p>for computing the square root of two, because we know that its root is the number we want. If we &#8220;shift this function up&#8221; by adding the number four to it, we get<\/p>\n\n\n\n\\[f(x)=x^2+2\\]\n\n\n\n<p>and there isn&#8217;t any (real) root any more. So, let&#8217;s try Newton&#8217;s method on this and see what happens. The function and its derivative look like this:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"557\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-no-root.jpg\" alt=\"Having no root of the function with Newton's Method: Overview\" class=\"wp-image-132\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-no-root.jpg 557w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-no-root-300x238.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-no-root-20x16.jpg 20w\" sizes=\"(max-width: 557px) 100vw, 557px\" \/><\/figure><\/div>\n\n\n\n<p>Here we can clearly see that the function (in blue) doesn&#8217;t even come near zero. Its derivative is problematic in the same fashion as above, i.e., there is a zero value for the slope at the origin. <\/p>\n\n\n\n<p>I started Newton&#8217;s method at \\(x_0=1.8\\) and gave it 10 steps to try and find a solution (remember that this was enough in the square-root-finding example above), but (obviously), the algorithm just keeps wandering around:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"545\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-no-root-10.jpg\" alt=\"Searching for a function's non-existing root with Newton's Method: Search with 10 steps\" class=\"wp-image-133\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-no-root-10.jpg 545w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-no-root-10-300x243.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-no-root-10-20x16.jpg 20w\" sizes=\"(max-width: 545px) 100vw, 545px\" \/><\/figure><\/div>\n\n\n\n<p>In order to see immediately that just using more steps doesn&#8217;t (because it cannot) help, here is the same plot again, but this time with ten times as many possible steps, i.e. 100 maximum iterations:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"545\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-100.jpg\" alt=\"Searching for a function's non-existing root with Newton's Method: Search with 100 steps\" class=\"wp-image-134\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-100.jpg 545w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-100-300x243.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-100-20x16.jpg 20w\" sizes=\"(max-width: 545px) 100vw, 545px\" \/><\/figure><\/div>\n\n\n\n<p>So, there are 100 tangents in this picture (or maybe not all of them actually show up between \\(x=-2\\) and \\(x=2\\). You can see a lot of them, that&#8217;s good enough. So, if your Newton-Raphson iteration does not converge and you don&#8217;t know why, this case is one possibility.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-5-newtons-method-running-away-into-an-asymptotic-region-of-the-function\"><\/span>Example 5: Newton&#8217;s method running away into an asymptotic region of the  function<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Another possible failure of Newton&#8217;s method happens, when there is an asymptotic region in the function, e.g., the function falls off monotonously towards positive infinity, but doesn&#8217;t reach zero. What? Let&#8217;s look at a concrete example. We&#8217;ll use the function<\/p>\n\n\n\n\\[f(x)=x \\exp(-x)\\]\n\n\n\n<p>which has such an asymptotic region towards positive infinity. Here it is plotted together with its derivative:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"577\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-asymptotic.jpg\" alt=\"Running away into the asymptotic region of the function with Newton's Method: Overview\" class=\"wp-image-137\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-asymptotic.jpg 577w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-asymptotic-300x230.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-asymptotic-20x15.jpg 20w\" sizes=\"(max-width: 577px) 100vw, 577px\" \/><\/figure><\/div>\n\n\n\n<p>We can see clearly that the function actually does have a root, which the Newton-Raphson method can also find, but only if it starts at a suitable point. Our choice of  \\(x_0=1.8\\), for example lies near the maximum of the function, more precisely, to the right of the maximum. If a search for a root with Newton&#8217;s method starts there, the following happens:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"577\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic.jpg\" alt=\"Searching for a function's oot with Newton's Method in an asymptotic region of the function\" class=\"wp-image-138\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic.jpg 577w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic-300x231.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic-20x15.jpg 20w\" sizes=\"(max-width: 577px) 100vw, 577px\" \/><\/figure><\/div>\n\n\n\n<p>We observe that the slope to the right of the maximum is always negative. So, once the method starts there, the \\(x_i\\) can only move to larger and larger numbers and their sequence actually diverges spectacularly.<\/p>\n\n\n\n<p>If, on the other hand, the search is started left of the maximum, say at \\(x_0=0.5\\), the method finds the root after 6 iteration steps. Here is the graph with the iteration path laid out by the red stars on the curves and the green Xs on the x-axis:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"571\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic-avoided.jpg\" alt=\"Searching for a function's root with Newton's Method avoiding an asymptotic region of the function\" class=\"wp-image-140\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic-avoided.jpg 571w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic-avoided-300x233.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-asymptotic-avoided-20x16.jpg 20w\" sizes=\"(max-width: 571px) 100vw, 571px\" \/><\/figure><\/div>\n\n\n\n<p>I should mention here that converging towards a solution of zero (as it happens here) is a little problematic when defined via a relative difference. Once the level of precision of the root is such that it appears as zero in the computer&#8217;s memory, a division by zero is encountered. However, meaningful values are produced on the way to this failure, so we can still feel content about having avoided the asymptotic trap of this function.<\/p>\n\n\n\n<p>For this case, I show the third kind of plot again (like in the first example), where the number of iteration steps needed for a solution is added to the overview. Here it is:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"563\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-asymptotic.jpg\" alt=\"Searching for a function's root with Newton's Method avoiding and falling into an asymptotic region of the function, depending on starting point\" class=\"wp-image-141\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-asymptotic.jpg 563w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-asymptotic-300x236.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-asymptotic-20x16.jpg 20w\" sizes=\"(max-width: 563px) 100vw, 563px\" \/><\/figure><\/div>\n\n\n\n<p>The red dots show the (obviously discrete) values for the number of necessary iterations, if Newton&#8217;s method is started at this value for \\(x_0\\). Keep in mind that there is a maximum number of iterations in my code in order to avoid an infinite loop when the algorithm fails to converge and stop. In this plot I set that maximum number to 20. <\/p>\n\n\n\n<p>We can clearly see that a plateau at 20 appears at the maximum value of our function and is there for the entire asymptotic region of the exponential tail. To the left of the maximum of \\(f\\), there is our usual stuff: fewer iterations closer to the root.<\/p>\n\n\n\n<p>To summarize this example: if you see your values for the \\(x_i\\) run away in one direction, maybe that&#8217;s the fault of an asymptotic region that your function shows there.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-6-newtons-method-oscillating-between-two-regions-forever\"><\/span>Example 6: Newton&#8217;s method oscillating between two regions forever<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Another possibility for Newton&#8217;s method to get stuck in an infinite search pattern is an oscillation. In particular, in such a case the iteration procedure with the tangents jumps between two regions, whose slopes &#8220;point at each other&#8221;. Now what is that supposed to mean? Let&#8217;s check out an example.<\/p>\n\n\n\n<p>A possible function that will show us this behavior is<\/p>\n\n\n\n\\[f(x)=-\\cos\\left(|x|+\\frac{\\pi}{4}\\right) + 0.8\\]\n\n\n\n<p>whose first derivative is<\/p>\n\n\n\n\\[f'(x)=\\sin\\left(|x|+\\frac{\\pi}{4}\\right)\\; \\text{sgn}(x)\\]\n\n\n\n<p>This is still hard to imagine just inside your head, so let me show you the graph of these two:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"562\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-oscillation.jpg\" alt=\"Oscillating between regions of the function with Newton's Method: Overview\" class=\"wp-image-146\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-oscillation.jpg 562w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-oscillation-300x236.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-oscillation-20x16.jpg 20w\" sizes=\"(max-width: 562px) 100vw, 562px\" \/><\/figure><\/div>\n\n\n\n<p>The interesting region for our purposes is in the middle, where the curvature is positive. Any Newton search in there will get trapped and cannot get out, for example:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"574\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-oscillation.jpg\" alt=\"Searching for a function's root with Newton's Method being trapped oscillating between two (adjacent) regions of the function\" class=\"wp-image-147\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-oscillation.jpg 574w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-oscillation-300x232.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-oscillation-20x15.jpg 20w\" sizes=\"(max-width: 574px) 100vw, 574px\" \/><\/figure><\/div>\n\n\n\n<p>What does that mean for the convergence regions of Newton&#8217;s method for this function? There is a region in the middle, around the origin, where the method fails. Outside of that it works most of the time, but fails, where the one of the steps in the iteration leads into the &#8220;oscillation trap&#8221; in the center:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"546\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-oscillation.jpg\" alt=\"Searching for a function's root with Newton's Method avoiding and falling into an oscillating region of the function, depending on starting point\" class=\"wp-image-148\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-oscillation.jpg 546w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-oscillation-300x243.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-oscillation-20x16.jpg 20w\" sizes=\"(max-width: 546px) 100vw, 546px\" \/><\/figure><\/div>\n\n\n\n<p>This may seem like an untypical case due to the function construction with the absolute value. However, the absolute value is not as uncommon as we would perhaps think or like. <\/p>\n\n\n\n<p>So if you see your Newton method&#8217;s points oscillate (in sign or otherwise), then something like the case shown here might be happening.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-7-newtons-method-fails-for-roots-rising-slower-than-a-square-root\"><\/span>Example 7: Newton&#8217;s method fails for roots rising slower than a square root<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>There are roots to which Newton&#8217;s method does not converge, but it diverges away from. Why? Because the function&#8217;s slope and curvature at the root produce iteration steps that go further and further away from the root, even if the initial guess is chosen very close to it.<\/p>\n\n\n\n<p>The &#8220;border&#8221; behavior of the function in question for this to happen is that of a square root, i.e., if \\(f(x)\\) at the root rises more slowly than<\/p>\n\n\n\n<p>\\[f(x)=\\sqrt{x}\\]<\/p>\n\n\n\n<p>then Newton&#8217;s method diverges away from the root. Let&#8217;s try this by investigating the cubic-root function:<\/p>\n\n\n\n<p>\\[f(x)=\\sqrt[3]{x}\\]<\/p>\n\n\n\n<p>In this case, the derivative is<\/p>\n\n\n\n<p>\\[f'(x)=\\frac{1}{3 x^{2\/3}}\\]<\/p>\n\n\n\n<p>and the iteration step becomes<\/p>\n\n\n\n<p>\\[x_{i+1}=x_i &#8211; 3x_i=-2x_i\\;.\\]<\/p>\n\n\n\n<p>Now, for this example we know that the root is located at \\(x=0\\). That means that, looking at the iteration step, we see how the \\(x_i\\) alternates in sign and grows in size at each step, i.e., it moves away from the root at zero.<\/p>\n\n\n\n<p>Numerically, in this particular example, the initial guess gets multiplied by a factor of negative two at each iteration step, growing geometrically beyond all bounds. As a result, running this as a numerical exercise can only help to expose missing maximum numbers of iteration, or producing numerical overflows.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-8-newtons-method-for-the-arctangent-function\"><\/span>Example 8: Newton&#8217;s method for the arctangent function<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>A similar example, but with a region of convergence in the center, is the arctangent function:<\/p>\n\n\n\n<p>\\[f(x)=\\arctan(x)\\]<\/p>\n\n\n\n<p>with the derivative<\/p>\n\n\n\n<p>\\[f'(x)=\\frac{1}{x^2+1}\\]<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"562\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-overview-arctan.jpg\" alt=\"Investigating the arctan function with Newton's Method: Overview\" class=\"wp-image-295\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-overview-arctan.jpg 562w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-overview-arctan-300x237.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-overview-arctan-20x16.jpg 20w\" sizes=\"(max-width: 562px) 100vw, 562px\" \/><\/figure><\/div>\n\n\n\n<p>In this case the iteration step becomes<\/p>\n\n\n\n<p>\\[x_{i+1}=x_i &#8211; (x_i^2+1) \\arctan(x_i)\\;.\\]<\/p>\n\n\n\n<p>Now, for choices of \\(x_0\\) close enough to the origin, we get a converging pattern, e.g., for \\(x_0=1\\):<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"562\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1.jpg\" alt=\"Investigating the arctan function with Newton's Method: Starting a search from initial guess 1.0\" class=\"wp-image-296\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1.jpg 562w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1-300x237.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1-20x16.jpg 20w\" sizes=\"(max-width: 562px) 100vw, 562px\" \/><\/figure><\/div>\n\n\n\n<p>However, if we start a bit further away from the origin, say at 1.5, Newton&#8217;s method diverges spectacularly:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"562\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1.5.jpg\" alt=\"Investigating the arctan function with Newton's Method: Starting a diverging search from initial guess 1.5\" class=\"wp-image-297\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1.5.jpg 562w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1.5-300x237.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/10\/newton-method-example-plot-search-arctan-1.5-20x16.jpg 20w\" sizes=\"(max-width: 562px) 100vw, 562px\" \/><\/figure><\/div>\n\n\n\n<p>Here, the method oscillates and diverges away from the origin, producing a number larger than the computer can handle in less than 20 steps.<\/p>\n\n\n\n<p>The boundary between these two regions (of convergence vs. divergence) can be found by solving the equation \\(x_{i+1}=-x_i\\), since the resulting sequence is alternating in sign. We find<\/p>\n\n\n\n<p>\\[x_i &#8211; (x_i^2+1) \\arctan(x_i)=-x_i\\]<\/p>\n\n\n\n<p>and, simplifying the equation,<\/p>\n\n\n\n<p>\\[\\arctan (x_i)=\\frac{2x_i}{x_i^2+1}\\]<\/p>\n\n\n\n<p>whose solution is approximately \\(1.39175\\). If the absolute value of the initial guess is larger than this number, Newton&#8217;s method diverges, for smaller absolute values of the initial guess, it converges.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-9-a-couple-of-roots-to-choose-from-for-newtons-method\"><\/span>Example 9: A couple of roots to choose from for Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Among all these examples for the use of Newton&#8217;s method to find roots of a function, we haven&#8217;t seen the most usual yet. So far, we&#8217;ve had cases with one root, no root, but not several roots. We&#8217;ve seen the method fail, but sometimes, the problem is not that it fails, but that it fails to find the solution that we want it to find. Sounds complicated? It can be. Here is an example. <\/p>\n\n\n\n<p>For this purpose, a polynomial is kind of ideal: it is easily constructed, can have several roots at specific locations, and derivatives are simple to get, too. So let&#8217;s use<\/p>\n\n\n\n\\[f(x)=(x+2)(x+1.5)(x-0.5)(x-2)\\]\n\n\n\n<p>with the first derivative<\/p>\n\n\n\n\\[f'(x)=(x+1.5)(x-0.5)(x-2)+(x+2)(x-0.5)(x-2)+(x+2)(x+1.5)(x-2)+(x+2)(x+1.5)(x-0.5)\\]\n\n\n\n<p>These two look like this:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"549\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-multiple-roots.jpg\" alt=\"Investigating a function with several roots with Newton's Method: Overview\" class=\"wp-image-150\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-multiple-roots.jpg 549w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-multiple-roots-300x242.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-multiple-roots-20x16.jpg 20w\" sizes=\"(max-width: 549px) 100vw, 549px\" \/><\/figure><\/div>\n\n\n\n<p>One can clearly see the four roots in the function, and that the derivative has three zeros, where Newton&#8217;s method will stop, if they are hit exactly. Other than that, there are no obvious problems. The convergence plot (which shows the number of iterations needed to arrive at a converged solution as red dots for each \\(x_0\\)) indicates exactly what I just stated: Nice convergence everywhere, except when hitting a zero in the first derivative.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"549\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-multiple-roots.jpg\" alt=\"Investigating a function with several roots with Newton's Method: Dependence on the starting point\" class=\"wp-image-151\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-multiple-roots.jpg 549w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-multiple-roots-300x242.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-multiple-roots-20x16.jpg 20w\" sizes=\"(max-width: 549px) 100vw, 549px\" \/><\/figure><\/div>\n\n\n\n<p>Still, it is an interesting case for the following reason: Let&#8217;s try and compute the root at \\(x=0.5\\). And let&#8217;s pretend that this function is actually unknown to us in detail (playing stupid for demonstration purposes). We somehow figure that the value \\(x_0=1\\) would be a good idea, so we start a run of Newton&#8217;s method with this initial guess. Here is the resulting search with all the tangents:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"549\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-1.jpg\" alt=\"Investigating a function with several roots with Newton's Method: Search starting with 1\" class=\"wp-image-152\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-1.jpg 549w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-1-300x242.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-1-20x16.jpg 20w\" sizes=\"(max-width: 549px) 100vw, 549px\" \/><\/figure><\/div>\n\n\n\n<p>Wow, this worked nicely! We found a root, and even better, we found our target! But what happens, if we don&#8217;t get so lucky, and someone had suggested \\(x_0=1.3\\)? Here is what happens:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"549\" height=\"442\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-13.jpg\" alt=\"Investigating a function with several roots with Newton's Method: Search starting with 1.3\" class=\"wp-image-153\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-13.jpg 549w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-13-300x242.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-13-20x16.jpg 20w\" sizes=\"(max-width: 549px) 100vw, 549px\" \/><\/figure><\/div>\n\n\n\n<p>Wow again, but in a different way! This search jumped to another root than the one closest to our starting point. And moving further to more positive starting values would bring it to the even further negative root at \\(x=-2\\). Then, just a little further, it would converge towards the most positive root at \\(x=2\\). So, you see that little differences in the initial guess can make quite a large difference in terms of where Newton&#8217;s method goes with its search pattern.<\/p>\n\n\n\n<p>As a consequence, we should always try to get as close as possible to a root with our initial guess!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-10-fractals-generated-with-newtons-method\"><\/span>Example 10: Fractals generated with Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Yes, indeed, there are fractals that can be generated by using Newton&#8217;s method. In short, the background is this: Although I kept stressing above at some places that we are looking for real roots, Newton&#8217;s method can be generalized to the complex plane (i.e., for complex numbers and functions). <\/p>\n\n\n\n<p>If a function is well-behaved enough also for complex arguments, then we can also extend the validity of its derivative away from the real axis. If this is too much information for you, don&#8217;t worry. Just imagine that you are computing a derivative of a function with two real variables instead of one.<\/p>\n\n\n\n<p>Now, how and why do fractals connect to Newton&#8217;s method and vice versa? The answer lies in the previous example. There, we have seen that Newton&#8217;s method can converge to different roots of a function from neighboring initial guesses \\(x_0\\). So far, we have only looked at the number of iterations it takes to arrive at a converged solution &#8211; any solution. <\/p>\n\n\n\n<p>But now imagine that we are color-coding each \\(x_0\\), depending on which root of \\(f\\) the iteration converges to. So, if there are four roots, like in the previous example, there would be four colors. And then, we run Newton&#8217;s method, starting from each point on a two-dimensional grid of <em>complex<\/em> numbers. <\/p>\n\n\n\n<p>We observe two things: to which root the method converges and how many iterations it takes to get there. The first (which root) is translated into the color of the starting point on the grid, and the second (how many iterations) can be used to additionally shade the color of that point.<\/p>\n\n\n\n<p>I know this is a bit much all at once, which is why I have written an entire article in addition to this one, just about Newton fractals. You can find it here: <a href=\"https:\/\/computingskillset.com\/solving-equations\/newton-fractals-explained-examples-and-python-code\/\">Newton Fractals Explained: Examples and Python Code<\/a>. Here is a sneak peek of such a fractal. The particular example is the one we used above as example number 7:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"837\" height=\"837\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-fractal-example-plot-sneak-peek.jpg\" alt=\"Example Plot of a Newton fractal\" class=\"wp-image-175\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-fractal-example-plot-sneak-peek.jpg 837w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-fractal-example-plot-sneak-peek-150x150.jpg 150w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-fractal-example-plot-sneak-peek-300x300.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-fractal-example-plot-sneak-peek-768x768.jpg 768w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-fractal-example-plot-sneak-peek-20x20.jpg 20w\" sizes=\"(max-width: 837px) 100vw, 837px\" \/><\/figure>\n\n\n\n<p>As you can see, this picture uses only color information and not shading &#8211; I leave the latter to the in-depth article linked above.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"example-11-finding-roots-with-higher-multiplicity-in-polynomials-with-newtons-method\"><\/span>Example 11: Finding roots with higher multiplicity in polynomials with Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Another typical example in connection with Newton&#8217;s method is the presence of multiple roots among the roots of a polynomial. So far, we have seen only simple roots in our examples. At this point, I&#8217;d like to show you one more with a double root. We&#8217;ll just reuse the function from example 7, but with a small difference:<\/p>\n\n\n\n\\[f(x)=(x+2)(x+1.5)^2(x-0.5)(x-2)\\]\n\n\n\n<p>Note that the second factor is squared now, which means that the root of this polynomial at \\(x=-1.5\\) is now double. The first derivative reflects this as well and becomes<\/p>\n\n\n\n\\[f'(x)=(x+1.5)^2(x-0.5)(x-2)+2(x+2)(x+1.5)(x-0.5)(x-2)+(x+2)(x+1.5)^2(x-2)+(x+2)(x+1.5)^2(x-0.5)\\]\n\n\n\n<p>The graphs of our function and its first derivative look similar to the one above in example 7, but a bit different:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"558\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-double-root.jpg\" alt=\"Investigating a function with several roots and one double (multiple) root with Newton's Method: Overview\" class=\"wp-image-160\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-double-root.jpg 558w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-double-root-300x239.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-overview-double-root-20x16.jpg 20w\" sizes=\"(max-width: 558px) 100vw, 558px\" \/><\/figure><\/div>\n\n\n\n<p>Admittedly, it is hard to see the region around the double root, so we&#8217;ll go right into the search plot. Starting at \\(x_0=-1\\), we arrive at the desired solution:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"560\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-double-root.jpg\" alt=\"Investigating a function with multiple roots with Newton's Method: Search starting with -1\" class=\"wp-image-161\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-double-root.jpg 560w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-double-root-300x238.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-search-double-root-20x16.jpg 20w\" sizes=\"(max-width: 560px) 100vw, 560px\" \/><\/figure><\/div>\n\n\n\n<p>Here one can see the graph of the function better around the double root. We also see how the tangents, stars and Xs are moving towards the solution. What isn&#8217;t evident from this figure, but is important and the reason for this example, is the speed of convergence towards the double root as compared to a single root.<\/p>\n\n\n\n<p>For this, our usual plot with the number of necessary iterations as a function of the starting point comes in handy. Here it is:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"558\" height=\"444\" src=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-double-root.jpg\" alt=\"Investigating a function with several roots and one double (multiple) root with Newton's Method: Convergence speed and dependence on initial guess\" class=\"wp-image-162\" srcset=\"https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-double-root.jpg 558w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-double-root-300x239.jpg 300w, https:\/\/computingskillset.com\/wp-content\/uploads\/2019\/09\/newton-method-example-plot-initial-guess-dependence-double-root-20x16.jpg 20w\" sizes=\"(max-width: 558px) 100vw, 558px\" \/><\/figure><\/div>\n\n\n\n<p>Apart from the occasional jitter around the zeros of the first derivative, there are two clearly different regions with regard to the number of necessary iterations. The region around the double root at \\(x=-1.5\\) and everything else, which we can explicitly call &#8220;regions around simple roots&#8221;.<\/p>\n\n\n\n<p>In particular, we see that around 30 iterations are needed to arrive at a converged result for the double root, while 5 or 6 suffice to get to a simple root with the same relative precision. <\/p>\n\n\n\n<p>The reason is theoretical, namely that Newton&#8217;s method usually converges rather well (quadratically), but not for multiple roots like our double example here. In principle, you can modify the Newton-Raphson algorithm, but in order to do that properly, you&#8217;d have to know the multiplicity of the root. So, in practice this is mostly not as helpful as we would hope.<\/p>\n\n\n\n<p>Let&#8217;s just keep in mind that if Newton&#8217;s method converges unusually slow, there may be a multiple root behind that behavior.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"python-code-to-generate-the-solutions-and-figures-for-these-examples-for-newtons-method\"><\/span>Python code to generate the solutions and figures for these examples for Newton&#8217;s method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Finally, in order for you to have some concrete starting point to learn more and to play around with Newton&#8217;s method and these examples, here is the python code that I used to do the calculations and produce the images. It comes as is and without any warranty, so use at your own risk.<\/p>\n\n\n\n<p>If you&#8217;d like to re-use or share this code somewhere else, you can do that as well &#8211; please read <a href=\"https:\/\/computingskillset.com\/notes-on-my-code-examples\/\">my notes on my code examples<\/a>. That being said, have fun!<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"classic\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">#!\/usr\/bin\/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCreated on Tue Sep 25 05:52:47 2019\n\nNewton-Raphson method code. \nSolves the method for a given initial guess \nas well as tests the entire x grid as initial guesses\nProduces three figures:\n    function overview\n    tangents for chosen initial guess\n    efficiency of other initial guesses\nIncludes several presets for function, but is easily extendable\n\n@author: Andreas Krassnigg, ComputingSkillSet.com\/about\/\n\nThis work is licensed under a \nCreative Commons Attribution-ShareAlike 4.0 International License.\nMore information: https:\/\/creativecommons.org\/licenses\/by-sa\/4.0\/\n\n\"\"\"\n\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n# function definitions to evaluate f and f' at x. Add your own as needed.\n\n# define the function and its derivative to investigate: \n# Demo function\ndef f(x):\n    valf = np.sin(x**2) - x**3 - 1\n    valdf = 2*x*np.cos(x**2) - 3*x**2\n    return (valf, valdf)\n\n# example 1: square root of two\ndef e1(x):\n    valf = x**2 - 2\n    valdf = 2*x\n    return (valf, valdf)\n\n# example 2: cube root of two\ndef e2(x):\n    valf = x**3 - 2\n    valdf = 3*x**2\n    return (valf, valdf)\n\n# example 3: seventh root of seven\ndef e3(x):\n    valf = x**7 - 7\n    valdf = 7*x**6\n    return (valf, valdf)\n\n# example 4: function without roots\ndef e4(x):\n    valf = x**2 + 2\n    valdf = 2*x\n    return (valf, valdf)\n\n# example 5: function with asymptotic region\ndef e5(x):\n    valf = x*np.exp(-x) \n    valdf = -x*np.exp(-x) + np.exp(-x)\n    return (valf, valdf)\n\n# example 6: function with oscillation\ndef e6(x):\n    valf = -np.cos(np.absolute(x)+np.pi\/4) + 0.8\n    valdf = np.sin(np.absolute(x)+np.pi\/4)*np.sign(x) \n    return (valf, valdf)\n\n# example 8: arctangent\ndef e8(x):\n    valf = np.arctan(x) \n    valdf = 1\/(x**2+1) \n    return (valf, valdf)\n\n# example 9: function with many roots\ndef e9(x):\n    valf = (x+2)*(x+1.5)*(x-0.5)*(x-2) \n    valdf = (x+1.5)*(x-0.5)*(x-2) + (x+2)*(x-0.5)*(x-2) +(x+2)*(x+1.5)*(x-2) + (x+2)*(x+1.5)*(x-0.5) \n    return (valf, valdf)\n\n# example 11: function with many roots, one multiple\ndef e11(x):\n    valf = (x+2)*(x+1.5)**2*(x-0.5)*(x-2) \n    valdf = (x+1.5)**2*(x-0.5)*(x-2) + 2*(x+2)*(x+1.5)*(x-0.5)*(x-2) +(x+2)*(x+1.5)**2*(x-2) + (x+2)*(x+1.5)**2*(x-0.5) \n    return (valf, valdf)\n\n\n\n\n# define parameters for plotting - adjust these as needed for your function\n    \n# define left and right boundaries for plotting    \n# for overview plots:\ninterval_left = -2.1\ninterval_right = 2.1\ninterval_down = -25\ninterval_up = 60\n\n# for search plot:\ninterval_left_search = -2.1\ninterval_right_search = -0.7\ninterval_down_search = -0.2\ninterval_up_search = 2\n\n# set number of points on x axis for plotting and initial-guess analysis\nnum_x = 1000\n\n# define the one initial guess you want to use to get a solution\nchosenx0 = -1\n\n# set desired precision and max number of iterations\nprec_goal = 1.e-10\nnmax = 100\n\n\n# the following defines function to solve and plot. Jump to end of code to choose function\n\n\n# define x grid of points for computation and plotting the function\nxvals = np.linspace(interval_left, interval_right, num=num_x)\n\n# create a list of startin points for Newton's method to loop over later in the code\n# we'll try all x values as starting points\npointlist = xvals\n\ndef solve_and_plot_newton(func):\n    \n    # begin plotting figure 1: search process with tangents\n    plt.figure()\n    \n    # plot the function f and its derivative\n    #fx, dfx = f(xvals)\n    fx, dfx = func(xvals)\n    plt.plot(xvals, fx, label='$f(x)$')\n    # plot the zero line for easy reference\n    plt.hlines(0, interval_left, interval_right)\n    \n     \n    # set limits for the x and y axes in the figure\n    # set x limits\n    plt.xlim((interval_left_search, interval_right_search))\n    # set y limits\n    plt.ylim((interval_down_search, interval_up_search))\n\n    #label the axes\n    plt.xlabel(\"$x$\", fontsize=16)\n\n    # initialize values for iteration loop\n    reldiff = 1\n    xi = chosenx0\n    counter = 0\n    \n    print('Starting Newton method at x0 =',chosenx0)\n    # start iteration loop\n    while reldiff > prec_goal and counter &lt; nmax:\n\n        # do the necessary computations\n        # get function value and derivative\n        fxi, dfxi = func(xi)\n        # compute relative difference\n        reldiff = np.abs(fxi\/dfxi\/xi)\n        # compute next xi\n        x1 = xi - fxi\/dfxi\n        # print numbers for use in convergence tables (csv format)\n        print('%i, %15.12f, %15.12f, %15.12f' % (counter+1, x1, fxi\/dfxi, reldiff))\n\n        # plot the tangents, points and intersections for finding the next xi\n        # plot xi on the axis\n        plt.plot(xi,0,'gx', markersize=16)\n        # plot a line from xi on the axis up to the curve\n        plt.vlines(xi, 0, fxi, colors='green', linestyles='dashed')\n    \n        # plot star on the curve where x=xi\n        plt.plot(xi,fxi,'r*', markersize=16)\n        # compute y values for tangent plotting\n        tangy = dfxi*xvals + fxi - dfxi*xi\n        # plot the tangent\n        plt.plot(xvals, tangy, 'r--')\n        # plot next xi on the axis\n        plt.plot(x1,0,'gx', markersize=16)\n\n        # trade output to input for next iteration step\n        xi = x1\n        # increase counter\n        counter += 1\n\n    # print test output\n    print(chosenx0,x1,counter,reldiff)\n    # remember counter for later plotting\n    counter_chosen = counter\n\n    plt.savefig('newton-method-example-plot-search.jpg', bbox_inches='tight')\n    #plt.savefig('newton-method-example-plot-demo.jpg', bbox_inches='tight', dpi=300)\n    \n    # close figure 1\n    plt.close()\n\n    # begin plotting figure 2: overview of function and its derivative\n    plt.figure()\n    \n    # plot the function f and its derivative\n    #fx, dfx = f(xvals)\n    fx, dfx = func(xvals)\n    plt.plot(xvals, fx, label='$f(x)$')\n    plt.plot(xvals, dfx, label=\"$f'(x)$\")\n    \n    # plot the zero line for easy reference\n    plt.hlines(0, interval_left, interval_right)\n    \n    #label the axes\n    plt.xlabel(\"$x$\", fontsize=16)\n#    plt.ylabel(\"$y(x)$\", fontsize=16)\n    \n    # set limits for the x and y axes in the figure\n    # set x limits\n    plt.xlim((interval_left, interval_right))\n    # we don't set y limits\n    plt.ylim((interval_down, interval_up))\n    \n    # add legends to the plot\n    plt.legend()\n    \n    # save the figure to an output file 1: overview of function and its derivative\n    plt.savefig('newton-method-example-plot-overview.jpg', bbox_inches='tight')\n    \n    # carry on plotting figure 3: overview plus\n    # loop over all starting points defined in pointlist\n    for x0 in pointlist:\n        # initialize values for iteration loop\n        reldiff = 1\n        xi = x0\n        counter = 0\n        # start iteration loop\n        while reldiff > prec_goal and counter &lt; nmax:\n    \n            # get the number of necessary iterations at that particular x0\n            # compute relative difference\n            fxi, dfxi = func(xi)\n            reldiff = np.abs(fxi\/dfxi\/xi)\n            # compute next xi\n            x1 = xi - fxi\/dfxi\n#            print('%i, %15.12f, %15.12f, %15.12f' % (counter+1, x1, fxi\/dfxi, reldiff))\n            # trade output to input for next iteration step\n            xi = x1\n            # increase counter\n            counter += 1\n    \n        # plot the number of necessary iterations at that particular x0\n        plt.plot(x0,counter,'r.', markersize=5)\n#        plt.plot(x0,counter,'r.', markersize=5, label=\"# of iterations\")\n        # print test output\n#        print(x0,x1,counter,reldiff)\n        \n    plt.plot(chosenx0,counter_chosen,'r.', markersize=5, label=\"# of iterations\")\n    # add legends to the plot\n    plt.legend()\n    # save the figure to an output file\n    plt.savefig('newton-method-example-plot-initial-guess-dependence.jpg', bbox_inches='tight')\n    #plt.savefig('newton-method-example-plot-demo.jpg', bbox_inches='tight', dpi=300)\n    \n    plt.close()\n\n\n# call the solution function for func of your choice\nsolve_and_plot_newton(e9)\n<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"further-information-about-the-newton-raphson-method\"><\/span>Further information about the Newton-Raphson method<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>There is quite a bit to explain, when it comes to the Newton-Raphson method, or Newton&#8217;s method. If you liked these examples, but need more information and in-depth explanations of the methods step by step, then head right over to my articles <a href=\"https:\/\/computingskillset.com\/solving-equations\/the-newton-raphson-method-explained-details-pictures-python-code\/\">Newton\u2019s Method Explained: Details, Pictures, Python Code<\/a> and <a href=\"https:\/\/computingskillset.com\/solving-equations\/how-to-find-the-initial-guess-in-newtons-method\/\">How to Find the Initial Guess in Newton\u2019s Method<\/a>. <\/p>\n\n\n\n<p>If you enjoy cool pictures, complex numbers, and even more of Newton&#8217;s method, check out my article <a href=\"https:\/\/computingskillset.com\/solving-equations\/newton-fractals-explained-examples-and-python-code\/\">Newton Fractals Explained: Examples and Python Code<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Newton&#8217;s method for numerically finding roots of an equation is most easily understood by example. At least, I learn more easily from examples. So, perhaps you do, too. In this<\/p>\n","protected":false},"author":1,"featured_media":173,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/posts\/93"}],"collection":[{"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/comments?post=93"}],"version-history":[{"count":38,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/posts\/93\/revisions"}],"predecessor-version":[{"id":299,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/posts\/93\/revisions\/299"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/media\/173"}],"wp:attachment":[{"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/media?parent=93"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/categories?post=93"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/computingskillset.com\/wp-json\/wp\/v2\/tags?post=93"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}