python libraries

[robmio]

Goodmorning everyone.
In the HP PRIME calculator, how do you import python libraries? For example: how do I import the scipy library, or the numpy library?
Best regards, Roberto.

[jonmoore]

Standard libraries aren't compatible AFAIK, but KhiCAS includes facsimiles of Numpy, Scipi and MatPlotLib so maybe Bernard might be willing to share them with the HP team (not that MatPlotLib is required).

[parisse]

Inside Khicas, Numpy compatibility is provided by a Pythons script file that translate some Numpy commands into linalg commands (linalg is a native library I wrote for MicroPython). Unfortunately, Numpy would not be compatible because HP Micropython version is too old, version 1.12 at least is required. My personal opinion about Numpy is that it's more complicated to use than call natively Prime or Xcas commands.
Matplotlib compatibility inside KhiCAS is done using a native graphic library I wrote for Micropython. Matplotl requires that micropython is linked with giac, which also means that graphic knows how to draw on the Prime. I believe the best here is to get Casio matplotlib Python file and replace the basic graphic casioplot calls with HP Prime calls.

[cyrille de brébisson]

Hello,

Using the con kit, you can add files to an app (a python derivated app in this case).
Then you can use the files using import fileName

Of course, only pure python files will be supported. Modules that rely on a native library will not work as there is no native support in Prime.

If your aim is to do math, I would suggest that you use the build in calculator features, which are very well designed and work well together and work well with the calculator UI and keyboard.

Are they python libraries that would be beneficial to add to prime that I could add?

Cyrille

[jonmoore]

(04-22-2021 11:17 AM)cyrille de brébisson Wrote:  Hello,

Using the con kit, you can add files to an app (a python derivated app in this case).
Then you can use the files using import fileName

Of course, only pure python files will be supported. Modules that rely on a native library will not work as there is no native support in Prime.

If your aim is to do math, I would suggest that you use the build in calculator features, which are very well designed and work well together and work well with the calculator UI and keyboard.

Are they python libraries that would be beneficial to add to prime that I could add?

Cyrille

Numpy, Pandas, Scipi and Patsy would be my request but as Bernard mentions, some may lack compatibility due to the version of MicroPython that's currently being utilised on the Prime. On reflection, I also like Bernards idea of utilising his Casio customisation of MatPlotLib with HP draw calls as it would provide greater flexibility.

Plotly is my favourite math/sci/stats oriented graphics package but I'm not sure whether there's much within it that can be converted for calculator use cases.

I'm making an assumption that any math functions that are pure python will be more performant than math functions that involve roundtrips via PPL/CAS (this seemed to be evidenced by the TI Planet post).

[roadrunner]

Scipy has a lot of functions that are not in the Prime CAS like Lambert W and Airy. It would be nice to have them available.

-road

[toml_12953]

(04-22-2021 11:17 AM)cyrille de brébisson Wrote:  Are they python libraries that would be beneficial to add to prime that I could add?

Cyrille

It would help if we could import the time module so the monotonic time functions would be accessible.

[parisse]

(04-22-2021 01:07 PM)roadrunner Wrote:  Scipy has a lot of functions that are not in the Prime CAS like Lambert W and Airy. It would be nice to have them available.

-road

LambertW is available in giac and has it's own implentation. Airy_Ai and Bi are in giac but they are using GSL for numeric evaluation.

[John Keith]

(04-22-2021 11:17 AM)cyrille de brébisson Wrote:  Are they python libraries that would be beneficial to add to prime that I could add?

itertools. (Or are core libraries such as this included?)

[robmio]

Although the "HP PRIME" G2 calculator is very fast, when it calculates the integral to obtain the "Studentize q Rance distribution CDF" function, it takes about 33 seconds. Is there a way, with Python, to reduce the computation time of this function?

#cas
Studentized_Range_q_P_Value(q,k,ν):=
BEGIN
LOCAL funzione;
funzione:=(√(2*π)*k*ν^(ν/2))/
(Gamma(ν/2)*2^(ν/2-1))*∫(s^(ν-1)*
e^((-ν*s^2/2))/(√(2*π))*∫(e^((-z^2)/2)/
(√(2*π))*(normald_cdf(0,1,z+q*s)-
normald_cdf(0,1,z))^(k-1),z,
-(∞),+∞),s,0,+∞);
RETURN 1-funzione;
END;
#end


This is one of the reasons why I asked if HP PRIME can "import" the Scipy library, Numpy or scipy.integrate (see https://www.youtube.com/watch?v=lzp48-AzUO0).

Best regards, Roberto

[jonmoore]

(04-22-2021 03:07 PM)parisse Wrote:  LambertW is available in giac and has it's own implentation. Airy_Ai and Bi are in giac but they are using GSL for numeric evaluation.

Yet again I find myself salivating for the standalone version of XCAS/GIAC on the Prime. For those who feel we already have the majority of the meaningful stuff in the native CAS, let's not forget that your ace Longfloat library is also available, which beats Swissmicros claim of the DM42 as being the worlds most precise calculator (and by some margin too if one wishes)!

One of the reasons I still turn to the 50g so much is Longfloat (a different author I believe). But that really is the tortoise to the XCAS Longfloat hare on a modern ARM processor...

[parisse]

(04-22-2021 04:24 PM)robmio Wrote:  Although the "HP PRIME" G2 calculator is very fast, when it calculates the integral to obtain the "Studentize q Rance distribution CDF" function, it takes about 33 seconds. Is there a way, with Python, to reduce the computation time of this function?
...
This is one of the reasons why I asked if HP PRIME can "import" the Scipy library, Numpy or scipy.integrate (see https://www.youtube.com/watch?v=lzp48-AzUO0).

Best regards, Roberto

I don't think that any Python scientific library is faster than my own adaptive numeric integration algorithm with the same accuracy (for multiple integrals Monte-Carlo like methods will be much faster but much less accurate). You can try to optimize your call. For example if your bounds are infinity (something that is impossible with a non-CAS language like Python), the CAS will do a change of variable to have finite boundaries, and this might slow down the computation compared to taking reasonable finite boundaries, e.g. z=-10.0 to 10.0 in the inner integral. You could try:

Code:


f(q,s):=1/sqrt(2*π)* integrate(exp((-z^2)/2)*
(normald_cdf(0,1,z+q*s)-normald_cdf(0,1,z))^(k-1),z,-10.0,10.0);
p(k,ν,q):=(√(2*π)*k*ν^(ν/2))/(Gamma(ν/2)*2^(ν/2-1))/(√(2*π))*
integrate(s^(ν-1)*e^(-ν*s^2/2)*f(q,s),s,1e-6,5.0);

where you replace 1e-6 and 5.0 by something appropriate (I have no idea of this function and what arguments are expected).
Of course this will be much slower than a good implementation of a special function using for example series expansion for some range of arguments, and asymptotic expansion for other ranges, or Tchebyscheff polynomial approximations, etc.

And MicroPython is not Python, porting any (large) library to MicroPython to the Prime would require work. It's much easier to port a C library with the same functionnalities, for example blas+lapack instead of numpy.

[jonmoore]

(04-22-2021 06:58 PM)parisse Wrote:  And MicroPython is not Python, porting any (large) library to MicroPython to the Prime would require work. It's much easier to port a C library with the same functionnalities, for example blas+lapack instead of numpy.

I always understood MicroPython to be closely related to Cython but Cyrille did ask.

[parisse]

I should have said: MicroPython is not CPython. Cython is a tool for CPython, i.e. translating existing Python code to C for speed reasons and make the bindings. The reason behind is that Python interpreters are slow, for example compared to Javascript interpreters, and are really slow compared to native code. Cython is a way for Python programmers who do not want to learn a compiled language to improve the speed of their code. I'm not convinced it's a good idea, some developers like Python a lot because they have a lot of tools easily available, but the drawback is that complexity is hidden, and auto-translating code that has bad complexity to C will not make the complexity any better. And even if your code has the right complexity, it has probably a style that once translated will not be full optimized.

But anyway that's not what would have to be done here: numpy is a collection of CPython bindings to existing C code (blas+lapack). Porting it to MicroPython means a complete rewrite of the bindings.
Below is a partial listing of the code I'm using to emulate some Numpy commands. As you can see it is really MicroPython code, I don't think you can automatically get this from CPython code.

Code:


static mp_obj_t linalg_zerosones(size_t n_args, const mp_obj_t *args,int v0,int v1) {
  size_t n1=0,m1=0;
  if (MP_OBJ_IS_SMALL_INT(args[0]))
    n1= MP_OBJ_SMALL_INT_VALUE(args[0]);
  if (n_args==2 && MP_OBJ_IS_SMALL_INT(args[1]))
    m1= MP_OBJ_SMALL_INT_VALUE(args[1]);
  if (v0==0 && v1==1)
    m1=n1;
  mp_obj_t r = mp_obj_new_list(0, NULL);
  bool alea=v0==-1;
  for (int i=0;i<n1;++i){
    if (m1){
      mp_obj_t l = mp_obj_new_list(0, NULL);
      for (int j=0;j<m1;++j){
    mp_obj_list_append(l, mp_obj_new_float(alea?rand()/(RAND_MAX+1.0):(i==j?v1:v0)));    
      }
      mp_obj_list_append(r, l);
    }
    else
      mp_obj_list_append(r, mp_obj_new_float(alea?rand()/(RAND_MAX+1.0):v0));
  }
  return r;
}
static mp_obj_t linalg_zeros(size_t n_args, const mp_obj_t *args) {
  return linalg_zerosones(n_args,args,0,0);
}
MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(linalg_zeros_obj, 1, 2, linalg_zeros);
static mp_obj_t linalg_ones(size_t n_args, const mp_obj_t *args) {
  return linalg_zerosones(n_args,args,1,1);
}
MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(linalg_ones_obj, 1, 2, linalg_ones);
static mp_obj_t linalg_eye(size_t n_args, const mp_obj_t *args) {
  return linalg_zerosones(n_args,args,0,1);
}

...


static const mp_rom_map_elem_t linalg_locals_dict_table[] = {
    { MP_ROM_QSTR(MP_QSTR_pi), mp_const_float_pi },
    { MP_ROM_QSTR(MP_QSTR_matrix), (mp_obj_t) &linalg_matrix_obj },
    { MP_ROM_QSTR(MP_QSTR_arange), (mp_obj_t) &linalg_arange_obj },
    { MP_ROM_QSTR(MP_QSTR_linspace), (mp_obj_t) &linalg_linspace_obj },
    { MP_ROM_QSTR(MP_QSTR_dot), (mp_obj_t) &linalg_dot_obj },
    { MP_ROM_QSTR(MP_QSTR_cross), (mp_obj_t) &linalg_cross_obj },
    { MP_ROM_QSTR(MP_QSTR_solve), (mp_obj_t) &linalg_solve_obj },
    { MP_ROM_QSTR(MP_QSTR_peval), (mp_obj_t) &linalg_peval_obj },
    { MP_ROM_QSTR(MP_QSTR_horner), (mp_obj_t) &linalg_peval_obj },
    { MP_ROM_QSTR(MP_QSTR_mul), (mp_obj_t) &linalg_dot_obj },
    { MP_ROM_QSTR(MP_QSTR_transpose), (mp_obj_t) &linalg_transpose_obj },
    { MP_ROM_QSTR(MP_QSTR_rref), (mp_obj_t) &linalg_rref_obj },
    { MP_ROM_QSTR(MP_QSTR_egv), (mp_obj_t) &linalg_egv_obj },
    { MP_ROM_QSTR(MP_QSTR_eigenvects), (mp_obj_t) &linalg_egv_obj },
    { MP_ROM_QSTR(MP_QSTR_eig), (mp_obj_t) &linalg_eig_obj },
    { MP_ROM_QSTR(MP_QSTR_det), (mp_obj_t) &linalg_det_obj },
    { MP_ROM_QSTR(MP_QSTR_zeros), (mp_obj_t) &linalg_zeros_obj },
    { MP_ROM_QSTR(MP_QSTR_ones), (mp_obj_t) &linalg_ones_obj },
    { MP_ROM_QSTR(MP_QSTR_eye), (mp_obj_t) &linalg_eye_obj },
    { MP_ROM_QSTR(MP_QSTR_identity), (mp_obj_t) &linalg_eye_obj },
    { MP_ROM_QSTR(MP_QSTR_idn), (mp_obj_t) &linalg_eye_obj },
    { MP_ROM_QSTR(MP_QSTR_rand), (mp_obj_t) &linalg_rand_obj },
    { MP_ROM_QSTR(MP_QSTR_ranv), (mp_obj_t) &linalg_rand_obj },
    { MP_ROM_QSTR(MP_QSTR_ranm), (mp_obj_t) &linalg_rand_obj },
    { MP_ROM_QSTR(MP_QSTR_shape), (mp_obj_t) &linalg_shape_obj },
    { MP_ROM_QSTR(MP_QSTR_size), (mp_obj_t) &linalg_size_obj },
    { MP_ROM_QSTR(MP_QSTR_inv), (mp_obj_t) &linalg_inv_obj },
    { MP_ROM_QSTR(MP_QSTR_re), (mp_obj_t) &linalg_re_obj },
    { MP_ROM_QSTR(MP_QSTR_proot), (mp_obj_t) &linalg_proot_obj },
    { MP_ROM_QSTR(MP_QSTR_pcoeff), (mp_obj_t) &linalg_pcoeff_obj },
    { MP_ROM_QSTR(MP_QSTR_fft), (mp_obj_t) &linalg_fft_obj },
    { MP_ROM_QSTR(MP_QSTR_ifft), (mp_obj_t) &linalg_ifft_obj },
    { MP_ROM_QSTR(MP_QSTR_apply), (mp_obj_t) &linalg_apply_obj },
    { MP_ROM_QSTR(MP_QSTR_add), (mp_obj_t) &linalg_add_obj },
    { MP_ROM_QSTR(MP_QSTR_sub), (mp_obj_t) &linalg_sub_obj },
    { MP_ROM_QSTR(MP_QSTR_re), (mp_obj_t) &linalg_re_obj },
    { MP_ROM_QSTR(MP_QSTR_real), (mp_obj_t) &linalg_re_obj },
    { MP_ROM_QSTR(MP_QSTR_im), (mp_obj_t) &linalg_im_obj },
    { MP_ROM_QSTR(MP_QSTR_imag), (mp_obj_t) &linalg_im_obj },
    { MP_ROM_QSTR(MP_QSTR_conj), (mp_obj_t) &linalg_conj_obj },
    { MP_ROM_QSTR(MP_QSTR_abs), (mp_obj_t) &linalg_abs_obj },
};


static MP_DEFINE_CONST_DICT(linalg_locals_dict, linalg_locals_dict_table);

const mp_obj_type_t linalg_type = {
    { &mp_type_type },
    .name = MP_QSTR_linalg,
    .locals_dict = (mp_obj_t)&linalg_locals_dict
};


STATIC const mp_rom_map_elem_t mp_module_linalg_globals_table[] = {
    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR__linalg) },
    { MP_ROM_QSTR(MP_QSTR_pi), mp_const_float_pi },
    { MP_ROM_QSTR(MP_QSTR_matrix), (mp_obj_t) &linalg_matrix_obj },
    { MP_ROM_QSTR(MP_QSTR_arange), (mp_obj_t) &linalg_arange_obj },
    { MP_ROM_QSTR(MP_QSTR_linspace), (mp_obj_t) &linalg_linspace_obj },
    { MP_ROM_QSTR(MP_QSTR_dot), (mp_obj_t) &linalg_dot_obj },
    { MP_ROM_QSTR(MP_QSTR_cross), (mp_obj_t) &linalg_cross_obj },
    { MP_ROM_QSTR(MP_QSTR_solve), (mp_obj_t) &linalg_solve_obj },
    { MP_ROM_QSTR(MP_QSTR_peval), (mp_obj_t) &linalg_peval_obj },
    { MP_ROM_QSTR(MP_QSTR_horner), (mp_obj_t) &linalg_peval_obj },
    { MP_ROM_QSTR(MP_QSTR_mul), (mp_obj_t) &linalg_dot_obj },
    { MP_ROM_QSTR(MP_QSTR_transpose), (mp_obj_t) &linalg_transpose_obj },
    { MP_ROM_QSTR(MP_QSTR_rref), (mp_obj_t) &linalg_rref_obj },
    { MP_ROM_QSTR(MP_QSTR_egv), (mp_obj_t) &linalg_egv_obj },
    { MP_ROM_QSTR(MP_QSTR_eigenvects), (mp_obj_t) &linalg_egv_obj },
    { MP_ROM_QSTR(MP_QSTR_eig), (mp_obj_t) &linalg_eig_obj },
    { MP_ROM_QSTR(MP_QSTR_det), (mp_obj_t) &linalg_det_obj },
    { MP_ROM_QSTR(MP_QSTR_zeros), (mp_obj_t) &linalg_zeros_obj },
    { MP_ROM_QSTR(MP_QSTR_ones), (mp_obj_t) &linalg_ones_obj },
    { MP_ROM_QSTR(MP_QSTR_eye), (mp_obj_t) &linalg_eye_obj },
    { MP_ROM_QSTR(MP_QSTR_identity), (mp_obj_t) &linalg_eye_obj },
    { MP_ROM_QSTR(MP_QSTR_idn), (mp_obj_t) &linalg_eye_obj },
    { MP_ROM_QSTR(MP_QSTR_rand), (mp_obj_t) &linalg_rand_obj },
    { MP_ROM_QSTR(MP_QSTR_ranv), (mp_obj_t) &linalg_rand_obj },
    { MP_ROM_QSTR(MP_QSTR_ranm), (mp_obj_t) &linalg_rand_obj },
    { MP_ROM_QSTR(MP_QSTR_shape), (mp_obj_t) &linalg_shape_obj },
    { MP_ROM_QSTR(MP_QSTR_size), (mp_obj_t) &linalg_size_obj },
    { MP_ROM_QSTR(MP_QSTR_inv), (mp_obj_t) &linalg_inv_obj },
    { MP_ROM_QSTR(MP_QSTR_re), (mp_obj_t) &linalg_re_obj },
    { MP_ROM_QSTR(MP_QSTR_proot), (mp_obj_t) &linalg_proot_obj },
    { MP_ROM_QSTR(MP_QSTR_pcoeff), (mp_obj_t) &linalg_pcoeff_obj },
    { MP_ROM_QSTR(MP_QSTR_fft), (mp_obj_t) &linalg_fft_obj },
    { MP_ROM_QSTR(MP_QSTR_ifft), (mp_obj_t) &linalg_ifft_obj },
    { MP_ROM_QSTR(MP_QSTR_apply), (mp_obj_t) &linalg_apply_obj },
    { MP_ROM_QSTR(MP_QSTR_add), (mp_obj_t) &linalg_add_obj },
    { MP_ROM_QSTR(MP_QSTR_sub), (mp_obj_t) &linalg_sub_obj },
    { MP_ROM_QSTR(MP_QSTR_re), (mp_obj_t) &linalg_re_obj },
    { MP_ROM_QSTR(MP_QSTR_real), (mp_obj_t) &linalg_re_obj },
    { MP_ROM_QSTR(MP_QSTR_im), (mp_obj_t) &linalg_im_obj },
    { MP_ROM_QSTR(MP_QSTR_imag), (mp_obj_t) &linalg_im_obj },
    { MP_ROM_QSTR(MP_QSTR_conj), (mp_obj_t) &linalg_conj_obj },
    { MP_ROM_QSTR(MP_QSTR_abs), (mp_obj_t) &linalg_abs_obj },
};

STATIC const mp_obj_dict_t mp_module_linalg_globals = {
    .base = {&mp_type_dict},
    .map = {
        .all_keys_are_qstrs = 1,
        .is_fixed = 1,
        .used = MP_ARRAY_SIZE(mp_module_linalg_globals_table),
        .alloc = MP_ARRAY_SIZE(mp_module_linalg_globals_table),
        .table = (mp_map_elem_t*)mp_module_linalg_globals_table,
    },
};

const mp_obj_module_t mp_module_linalg = {
    .base = { &mp_type_module },
    .globals = (mp_obj_dict_t*)&mp_module_linalg_globals,
};

[jonmoore]

Bernard, out of interest. How much of Numpy and Scipi did you 'clone' for the Ndless version of XCAS/Giac? And on the subject of Cython/Cpython, don't the developers of SageMath use the optimisations you discuss in your last post to optimise certain aspects of SageMath inclusive of those that utilise Giac?

[parisse]

(04-23-2021 09:23 AM)jonmoore Wrote:  Bernard, out of interest. How much of Numpy and Scipi did you 'clone' for the Ndless version of XCAS/Giac?

It's hard to answer, because I do not know numpy and scipy myself. I just looked at some scripts for schools and tried to make them compatible. A little bit like Python syntax in Xcas, but I tested much less, because in the meantime I decided it was not worth the time required to have something more compatible. The guys who have imposed Python as the unique langage in French highschools are mostly closed-minded, they will probably never consider Xcas to be an alternative to Python for programming in math classes despite my efforts. And if are already using Xcas, it's much easier to run the native commands of Xcas than using all these Python modules.

Quote:And on the subject of Cython/Cpython, don't the developers of SageMath use the optimisations you discuss in your last post to optimise certain aspects of SageMath inclusive of those that utilise Giac?

Giac is interfaced to CPython with Cython, independently from Sage, you can import giacpy from Python if you install giacpy. I don't know if all C libraries are interfaced using cython inside Sagemath, probably.
My opinion is that Sagemath "Python as a common basis" choice is wrong, because if you want to use their code inside your code, then you must interface with Python. It means that C libraries inside Sagemaths can not talk together unless they have their own interface. Inside Sage, at the Python level, you can choose one algorithm or another one, but once you are in non Python code (e.g. Maxima code because Sage does not have calculus itself, or C code because Python is slow), you can not benefit from other parts of the whole system (for example your C code can not benefit from other C libraries included in Sage unless you wrote your own interface to these other C libraries, or Maxima code does not benefit from fast linear algebra). You must write Python code to benefit from Sage and it will be useful to Python programmers only.
I believe that the choice should have been to make C libraries talk together, because it does not make assumptions on the caller language.

[jonmoore]

(04-23-2021 10:54 AM)parisse Wrote:  I believe that the choice should have been to make C libraries talk together, because it does not make assumptions on the caller language.

100% agreement.