25MAR2017 SolveSys  Equation Library and Nonlinear Systems Solver

02052017, 10:51 PM
(This post was last modified: 03262017 03:27 AM by Han.)
Post: #1




25MAR2017 SolveSys  Equation Library and Nonlinear Systems Solver
25MARCH2017 (Ver. 1.000)
CHANGELOG
Download link at bottom of post; same link but zip file has been updated. NOTE: I am putting any further development (i.e. no new features) until summer; until then, any new update will only address bugs in the current version. Sorry, but I need to direct my focus on other things. Of course, I will still keep up with requests and relevant posts to this thread. ABOUT THE APP This app is intended to replicate and combine programs from the HP48 series of calculators for organizing equations and launching an equation solver. While there are a few usercreated programs/apps that help manage equations, most of them have the formulas builtin (hardcoded into the source code of the program/app). This app attempts to allow users to build their own equations library. The user can then also launch a numerical solver. Using the builtin Solve app may be adequate for some users, but it lacks the ability to handle more general systems. The current Solve app is quite deficient in that in requires the number of equations in a system be the same as the number of variables. Therefore, overdetermined or underdetermined systems cannot be solved numerically. Usually underdetermine systems have an infinite number of results, and a numerical solver should ideally try to return a solution that is as close to the initial guess as possible. For overdetermined systems, a solution usually does not exist. Most numerical solvers would return a "leastsquares" solution. This app implements the NewtonRaphson algorithm for solving a system of equations. In order to handle illconditioned, overdetermined, underdetermined, rankdeficient (etc.) Jacobian matrices, the SVD is used to determine the pseudoinverse of the Jacobian (the pseudoinverse coincides with the regular inverse of a matrix for nonsingular matrices). Users have reported that the app can handle complex values just fine. Minimal testing seems to confirm this. INSTALLATION AND USAGE (Please refer to the reference manual included in the zip file) REFERENCE MANUAL The reference manual will likely see another update in which all the images are updated to nicer, crisp images created by \( \LaTeX\ \) and pstricks. For now, though, I will leave them there so that there is at least some frame of reference, and for old time's sake. The equations will also likely get a face lift as the variables are updated to follow typical mathematical conventions (e.g. subscripts). While the HP Prime currently does not support subscripted variables, the correspondence between the calculator's variables and those in the reference manual should be obvious (e.g. \( v_0 \) vs. \( \tt{v0} \) ). If you notice any typos, please let me know. OTHER NOTES: There are two versions: a merged version (SVD merged into the source code for Equation Library) and the separated version. Does it make a difference which version I use? Yes, if you plan to work with under/over determined systems or systems whose Jacobian is either singular or near singular, it can make a major difference. PRIOR CHANGES: (08MAR2017 v0.900)
TODO
The current firmware seems to exhibit issues with CAS programs. If you should find the app not working suddenly, type restart; at the command line (NOTE: this will delete all CAS results) and "recompile" the app and/or the SVD program (simply open and close their files in the program editor). Turn the calculator off and then on to save the memory state. DOWNLOADS Version 1.000 (same file name, just redownload to get the update) Equation Library Prime.zip (Size: 540.99 KB / Downloads: 808) Graph 3D  QPI  SolveSys 

02092017, 08:42 PM
Post: #2




RE: SolveSys  Equation Library and Multiple Equations Solver
Hello, downloaded beta version. Thank you for your efforts on try to have a solvesys like HP50, one of the most important programs, I think.
First, a strange issue has happend sometimes: after inserting the equations, I push [Num] then OK, and it does not start to solve but it returns to the equation menu. ¿? I did a warmstart and it didn´t happen again, at the moment. The system of this thread gives different results from the Wolfram, I get A=1,27E10 and B=1,088E21. Wolfram: A: 3,675E10, B: 9,438€11 Perhaps too low values for iteration accuracy? http://www.hpmuseum.org/forum/thread767...t=solvesys Ok on this http://www.hpmuseum.org/forum/thread614...t=solvesys Ok on this http://www.hpmuseum.org/forum/thread604...t=solvesys Ok on this http://www.hpmuseum.org/forum/thread238...t=solvesys I´ll make more tests. 

02092017, 09:35 PM
(This post was last modified: 02092017 09:49 PM by Han.)
Post: #3




RE: SolveSys  Equation Library and Multiple Equations Solver
(02092017 08:42 PM)akmon Wrote: Hello, downloaded beta version. Thank you for your efforts on try to have a solvesys like HP50, one of the most important programs, I think. If I recall correctly, that pair of values results in a nearlysingular Jacobian (likely singular due to floating point issues). But getting there was likely due to a few other issues. The SVD program had a few bugs in it (update again, even if you already did so recently). The other issue is due to CAS corruption, for which the current workaround is to use the restart command in the CAS view and rebuild the SVD program by opening and closing its source file. I can still get both solutions to this system (in the 0.500 version) by setting x to 1e20 and F to 1e10. The F value I think should be selfexplanatory but the x labeling for the other tolerance may be a bit misleading. It is really a measure of the change in the x_i's relative to the norm of the previous solution. So if u want 4 digits of accuracy and your solution is in the order of 1e10, then you should try setting x to 1e16. May I can change this later on to "number of digits" rather than a "tolerance" value to reflect the true behavior of that value. Also, F should be set to be a 1e10 (or smaller) since 1.27e10 would be pretty close to the error itself (similar reasoning). When using (1,1) and (1,2) for initial values, I get the two separate solutions. Lastly, keep in mind that the (now prior) version was just Newton's method and is highly likely to wander off into "lala" land. In your case, you probably landed in a spot where the Jacobian is nearly singular (or in fact singular but not shown to be so due to typical floating point issues). You can check this by using svd2(ssJ)  make sure to do this in the CAS view. The "middle" vector lists the singular values. If any of them are close to zero, then the Jacobian is (likely) rankdeficient. The newer version has backtracking line search. I still need to tweak the exit conclusions (i.e. how to classify supposed solutions). I just updated the program; I appreciate your continued efforts to test the solver. EDIT: I should mention that I am able to get it to run on the real calculator after having tweaked a few spots where iferr blocks were used. It would seem that iferr and return calls are currently misbehaving so that repeated instances would lead to a warmstart. I worked around that issue by modifying the code to not rely on iferr. Graph 3D  QPI  SolveSys 

02102017, 12:18 AM
Post: #4




RE: SolveSys  Equation Library and Multiple Equations Solver
Thank you for the new release, I have also updated SVD. I have edited the solver settings with your advice, edited the solver settings, previous values [1,1], and after a few seconds I get A=3,36E5 ; B= 1,776€5, (100 iterations). After the results I started to solve again with these new initial values, and after a few seconds the result was correct. BUT, I don´t know why I cannot exit from the matrix environment. I can go to the solver menu, but no more, no Home, no CAS, strange, only happened once, tried to do the same again and no problem this time.
Edited solver parameters to 200 iterations and the result was correct. and if I assign [1,2] it shows "linesearch failed f> fold): yes lam 0,371136323793 IncrX=7,45499482206 If I use [1,1] or [0,0] or [1,0] and press OK, it returms to equation menu. ¿? By the way, the clocksand didn´t appear while solving with [1,1], I thought it hanged, but finished the calculus. And it seems that sometimes SVD does not exist. I edit the program or debug, and it works again. 

02102017, 12:30 AM
Post: #5




RE: SolveSys  Equation Library and Multiple Equations Solver
(02102017 12:18 AM)akmon Wrote: Thank you for the new release, I have also updated SVD. I have edited the solver settings with your advice, edited the solver settings, previous values [1,1], and after a few seconds I get A=3,36E5 ; B= 1,776€5, (100 iterations). After the results I started to solve again with these new initial values, and after a few seconds the result was correct. BUT, I don´t know why I cannot exit from the matrix environment. I can go to the solver menu, but no more, no Home, no CAS, strange, only happened once, tried to do the same again and no problem this time. It looks like you have memory corruption (referring to the bold text). The solving process is a bit slower in this version due to the backtracking. Before, every iteration step was a full Newton step in the opposite direction of the gradient. This could mean fewer iterations, and in other cases it could mean the Newton steps wander around in an unwanted direction. The backtracking line search technique takes smaller steps, but ensures that each step is a descent step toward either a 0 or a minimum. This is the tradeoff; perhaps I can make it optional to use line searches. As for the lockup issues: Try the following: 1) type restart into the CAS view 2) go into the memory browser, and look in the CAS Vars submenu; delete anything that is still left over in this submenu  make sure this is completely clean 3) turn the calculator off, and then back on (this presumably forces the memory to save its state into flash) 4) warmstart using [On][Symb] When the calculator warmstarts, it should rebuild all stored programs. If you see a message about installing SVD then open and immediately close the source file for SVD. Then try your solve. Graph 3D  QPI  SolveSys 

02102017, 07:27 AM
Post: #6




RE: SolveSys  Equation Library and Multiple Equations Solver
Did all the steps you suggested, but the results are exactly the same as the post I worte yesterday, the same behaviour (tests on my physical calculator). If you need any parameter about my settings, please let me know.


02102017, 04:46 PM
Post: #7




RE: SolveSys  Equation Library and Multiple Equations Solver
It looks like our equations are slightly different. If your equation is \( \left( \frac{A+B}{A} \right) ^2 = 8.377 \) then a guess of A=1 and B=2 would lead to a solution whereas A=2 and B=1 would lead to a failed line search.
If your equation is \( \left( \frac{A+B}{B} \right) ^2 = 8.377 \)  notice the denominator  then a guess of A=2 and B=1 would lead to a solution; A=1 and B=2 would lead to a failed line search (symmetry in the variables compared to the first case) For the rest of this post, let us assume you used the first form of (with A in the denominator). Assuming that our application is bugfree (doubtful at the moment), then A=2 and B=1 is a point for which a line search is going to fail. To see how this might come to be, let me explain the algorithm a bit. In the recent versions, we are additionally solving for the minimums of another function f = 1/2 (F.F). Notice that the zeros of F are guaranteed minimums of f (since it is a nonnegative function). However, not all minimums of f are zeros of F  no worries, we simply check whether \( \ F \ \) is sufficiently small enough). We are still using Newton's method, wherein we compute \( \Delta x = J^{1} \cdot F \) and obtain \( x_\text{new} = x_\text{old} + \Delta x \), however we scale the step \( \Delta x \) by \( \lambda \) so that we guarantee a decrease in \( \ F \ \). (J here represents the Jacobian of F). This is the line search portion of the algorithm. You can read about line searches here: http://reference.wolfram.com/language/tu...thods.html Basically, the line search tries to find a suitable \( \lambda \) that satisfies the strong Wolfe conditions (so that we have convergence). (In the link above, they call the scalar factor \( \alpha^* \) ). From the link: Quote:Typically, it only takes a few iterations to find a point satisfying the conditions. However, the line search is not always able to find a point that satisfies the conditions. Usually this is because there is insufficient precision to compute the points closely enough to satisfy the conditions, but it can also be caused by functions that are not completely smooth or vary extremely slowly in the neighborhood of a minimum. So in the end, A=2 and B=1 is just not a good guess (again, assuming A is the variable in the denominator of your equation involving 8.377). Graph 3D  QPI  SolveSys 

02102017, 07:13 PM
Post: #8




RE: SolveSys  Equation Library and Multiple Equations Solver
Understood. I have always made the examples using B as denominator, so if I insert A=1 and B=2 then an error message appears as yours. But with A=2 and B=1 I find the solution after 180 iterations
Please use the equations with B denominator and Guess with [0,0] or [1,0], it returns to equation menu again. As I have now in Verbose mode, I can read (slightly) that before going to equation menu, the program ends in creating the F vector. On virtual prime is even worse. It crashes with those guess values. 

02102017, 07:41 PM
Post: #9




RE: SolveSys  Equation Library and Multiple Equations Solver
(02102017 07:13 PM)akmon Wrote: On virtual prime is even worse. It crashes with those guess values. Wow, that is a huge crash! Graph 3D  QPI  SolveSys 

02112017, 02:09 AM
(This post was last modified: 02112017 02:13 AM by Han.)
Post: #10




RE: SolveSys  Equation Library and Multiple Equations Solver
I think to get around all these firmware issues, I will be
1) converting the svd code from CAS form into nonCAS form 2) then embedding it within the equation library / solver app This will mean no more corruption of the svd2 program (although a few variables must unfortunately remain on the cas side) and not having to update two separate programs. There is a tiny downside (possibly) since the home programs use BCD representation of floating points and the cas programs use binary representation. I am not sure how many bits are used on either side, but it could mean a tiny bit of accuracy loss switching out of cas form. Graph 3D  QPI  SolveSys 

02112017, 05:43 AM
(This post was last modified: 02112017 06:59 PM by Han.)
Post: #11




RE: SolveSys  Equation Library and Multiple Equations Solver
See if this version works any better. It will still misbehave in the event of something like you described above (and crash the VC), but this version no longer makes use of CAS programs to do the svd calculation. All of that is now builtin and part of the app as a whole. So you will need to delete your existing copy of svd2 as they are now part of the app .
For good measure, clean out your CAS variables to ensure none of the old stuff is still lying around, possibly causing collisions with their duplicate versions inside the app's code. eqlibsvd_10feb2017.txt (Size: 62.59 KB / Downloads: 36) Anyone interested in creating a nice icon for this app? Or perhaps creating the data file that would correspond to the equation library from the HP48 series? EDIT: The nonCAS version of svd2 appears to be slightly less accurate than the CAS version (off at about the 12th digit) (I'm guessing this is due to the floating point format) Graph 3D  QPI  SolveSys 

02122017, 11:19 PM
(This post was last modified: 02132017 12:26 AM by akmon.)
Post: #12




RE: SolveSys  Equation Library and Multiple Equations Solver
Now that SVD is included in the core of the app, it´s not only more conformtable for updating, but it avoids errors from "SVD not found". Well done. Buy the way, remember to update your first post.
I haved noticed that slightly less accuracy in our famous system (A= 3,4985E10, B=8,98E11), I have also discovered that if you are very accurate on your guess you´ll reach the exact solution, but it is too much to suppose and this system of equations are very unlikely to appear in general terms. What still concern me is what causes that fatal crash on VC just only choosing from one guess to other. By the way, I´ve tried to solve a complex system. Always get bad guesses. How do I solve this? (2+i)*X+2*Y=1+7*i ; (1i)*X+i*Y=0 About icons, three examples. The first, represents the graphical purpose of this app. The second, a wink to the firs Solvesys, and the last, another wink to the first EQlib from HP48. 

02122017, 11:39 PM
Post: #13




RE: SolveSys  Equation Library and Multiple Equations Solver
(02122017 11:19 PM)akmon Wrote: Now that SVD is included in the core of the app, it´s not only more conformtable for updating, but it avoids errors from "SVD not found". Well done. Buy the way, remember to update your first post. This is exactly what I was referring to regarding the loss in accuracy. Using the CAS version, you get just a slight advantage in accuracy. (see http://www.hpmuseum.org/forum/thread2953.html) But you are right, such a system tends to be atypical due to the singularity at B=0 (assuming (((A+B)/B)^2\) ), or A=B Right now, you should only need to update the SVD program if there's a bug fix. However, you will still have to deal with the CAS corruption (esp. on the emulator). On the hardware, it should not occur unless you either 1) recompiled a program or 2) experience a warmstart. My posting the code as source code probably doesn't help with (1). The crash is due to the firmware's inability to handle undefined values in the Home view. The app is programmed as a "nonCAS" program. However, it still makes a few calls into the CAS. I hope that they will fix this in the next firmware. Quote:About icons, three examples. The first, represents the graphical purpose of this app. The second, a wink to the firs Solvesys, and the last, another wink to the first EQlib from HP48. Ahh, I do like the nod to either SolveSys or EQLIB. And compsystems' idea about using the old solve card as an icon is also a nice idea. Graph 3D  QPI  SolveSys 

02132017, 12:08 AM
(This post was last modified: 02142017 04:02 PM by Han.)
Post: #14




RE: SolveSys  Equation Library and Multiple Equations Solver
Equation Library data (partially completed). With the data below, you can get an idea of what the final version might look like. Please note that it is merely an example of a .lib file and is currently incomplete, or even wrong in some parts as I just recently learned.
If you would like a data file to test, enter the app and copy the list of lists below (easiest if done on emulator), and type: AFiles("Equation Library.lib"):=<insert copied list of lists below here> Code:
Graph 3D  QPI  SolveSys 

02132017, 12:39 PM
Post: #15




RE: SolveSys  Equation Library and Multiple Equations Solver
Utility to convert old library data to newer format:
Code: EXPORT UpdateLibDat() Graph 3D  QPI  SolveSys 

02142017, 02:11 AM
Post: #16




RE: SolveSys  Equation Library and Multiple Equations Solver
Here is another utility that will take your library data and convert it into a note, which you can then export into a text file. It assumes your library data has be converted up to the version used in 0.651
Code: EXPORT LibToNote() Graph 3D  QPI  SolveSys 

02142017, 08:06 PM
Post: #17




RE: SolveSys  Equation Library and Multiple Equations Solver
The icon


02142017, 11:21 PM
Post: #18




RE: SolveSys  Equation Library and Multiple Equations Solver
Downloaded equation library. I have choosen an easy example:
P=I^2*R P=V^2/R Suppose V and R are known variables: V=220 and R is 9 ohm. This example was able to be solved by MSOLVR fron HP48, because you a have a "concatenated" list of equations, solve P, then solve I. However the app can´t solve it. Tried various guesses and no way. Solvesys is able to work as MSOLVR if necesary? 

02152017, 12:16 AM
(This post was last modified: 02152017 07:20 PM by Han.)
Post: #19




RE: SolveSys  Equation Library and Multiple Equations Solver
(02142017 11:21 PM)akmon Wrote: Downloaded equation library. I have choosen an easy example: I am not quite sure what you mean by not being able to solve it. If you want it to solve for P and I while fixing the values of V and R, then you will need to tell the solver that the values of V and R are constant. To the far right of the variables are check boxes to set them to be constants (when checked) or variables (when unchecked). If you leave them as variables, then you are actually solving an underdetermined system (4 variables with only 2 equations  there are an infinite number of solutions). On the other hand, setting V and R to be constants gives me the solution P=5377.7778 and I=24.4444 (2 variables, 2 equations  if a solution exists, it will be unique). I used a guess of 100, 100. I changed my tolerances because the values supplied (220 and 9) suggest we only need to be within 5 or so decimal places for a reasonable answer (unlike the other problem where the solution itself was in the order of 10^10 or 10^11). Please keep in mind that the conclusion from the app is all based on your tolerances. It stops the iterations if the x_i's are within the tolerance you specify (it's actually not a tolerance, but a check on whether the digits up to a certain order have stopped changing). Then it checks to see if the output values are also within (another) tolerance, concluding a zero if yes. And if not it checks to see if perhaps a minimum (or extremum in general) may have been found. These last two are the only possible, valid conclusions since a "tiny" (or no) change in the x_i's suggests that the gradient is nearly 0 (which occurs at extrema). Here, "tiny" is relative to the tolerance you set. But again, these conclusions are based on your tolerances. So if you set them too large, then it may conclude incorrectly. Set them too small, and you may never see convergence in some situations (not enough precisions to meet the desired tolerance) despite the values actually converging. That said, I have been considering implementing a hybrid solver just as described by the HP patent. Since the Prime has access to a CAS, it should not be too hard to implement a secondary solver (or maybe this could be the primary solver) in which a formula having only one parameter that is unknown is used to determine the remaining parameter, repeating as much as necessary while cycling through the remaining formulas. EDIT: Ahh, ok. I downloaded the version listed in the first post, and am seeing the misbehavior. There's a single line in the function ssEvalF that was modified and somehow did not make it into this revision (this is what I get for not using a CVS and developing on multiple copies over multiple machines). In the meantime, change the line ssCAS(ssCurSys(4,j) + ":=" + vec(j)); to become ssCAS(ssCurSys(4,ssSolveVars(j)) + ":=" + vec(j)); within the function ssEvalF(vec) I found a few more bugs that I need to iron out in the pinv() routine (for degenerate cases like m x 1 or 1 x n matrices) but I should have an update up soon. EDIT 2: 0.652 should take care of this, now. SVD was also updated. Graph 3D  QPI  SolveSys 

02152017, 10:41 PM
Post: #20




RE: SolveSys  Equation Library and Multiple Equations Solver
Ok, this is another thing. It solves correctly. It´s curious that if I insert R=8, the program founds a zero, but if I input R=9 it says "Bad guess", however the result is correct. The clue is F value. If it is very low you can assume that the solution is quite correct. I´ll make more tests.
By the way, this equation was pending of answer. I´ve read that the app allows to insert complex equations, I tried this: (2+i)*X+2*Y=1+7*i ; (1i)*X+i*Y=0 but I get a syntax error, don´t know to input the i value. 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)