Discussion:
Keeping OTB algorithm in qgis processing
(too old to reply)
Rashad Kanavath
2018-01-31 09:40:37 UTC
Permalink
Hello all,

I would like to check the pros and cons of keeping otb in qgis processing
like others. SAGA, GRASS, GDAL etc..

one or only argument, I found interesting is the fixes to otb plugin can be
done independent of qgis release and versions. But how does other
algorithms won't get this cool feature and only OTB?

One of the reasons why otb plugin got outdated was the separate maintenance
issue. otb team were not involved in maintaining this code
(processing/algs/otb) on a regular basis. And I don't think putting work on
otb team is interesting here. The plugin code works closely with qgis
processing with some specifics for running otb. So someone from otb team
needs to track changes in qgis code to keep users from running into
problems (installation or running of plugins itself)

For one thing, this didn't worked out well in the past. All algorithms now
in qgis master is updated for qgis3 except for OTB because it's an external
plugin. Well, it not like other plugins it depends on qgis processing
plugin.

And due to this change code got left out by both teams. so the argument
that updates are independently managed is easy didn't worked here. the
problem is solved by creation of another one.

The old code has lot of hack among other issues. So fixing that is a
priority for otb. By that I mean no specific hacks and easy to maintain
like SAGA, GRASS etc...
With that change coming, are you folks interested to put back otb algo into
processing/algs/otb.?
--
Regards,
Rashad
Paolo Cavallini
2018-01-31 09:45:06 UTC
Permalink
Post by Rashad Kanavath
With that change coming, are you folks interested to put back otb algo
into processing/algs/otb.?
I am supportive of this, even though I understand the argument that
brought to the removal.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Rashad Kanavath
2018-01-31 09:50:59 UTC
Permalink
Post by Paolo Cavallini
Post by Rashad Kanavath
With that change coming, are you folks interested to put back otb algo
into processing/algs/otb.?
I am supportive of this, even though I understand the argument that
brought to the removal.
Thanks Paolo,
Can you give a link to that discussion?

All the best.
Post by Paolo Cavallini
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Regards,
Rashad
Paolo Cavallini
2018-01-31 10:04:47 UTC
Permalink
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion? 
I had a look to the archives, and I only found this:
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Victor Olaya
2018-01-31 10:17:54 UTC
Permalink
The original idea was to have most providers removed...which included
R, GRASS and SAGA as well. Finally, only certain providers were
removed, mostly those that require dependencies not provided with QGIS
(LiDAR tools, R...and OTB)

If maintaining the plugin is complicated and no work is done on that,
then definitely it is better to keep it as core. But we have to make
sure that the user knows that the provider by itself is useless, and
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea

My 2 cents
Post by Paolo Cavallini
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion?
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Alexander Bruy
2018-01-31 10:23:13 UTC
Permalink
Another reason to remove providers is that algorithms descriptions
were not updated and maintained. Even now both SAGA and GRASS
providers are not updated to latest stable versions of the corresponding
software.

We just simply does not have enough resources to maintain more than
600 algorithms from GRASS and SAGA. Situation with OTB, was even
worse, it requires special handling of the parameters.

I still think that Processing should be in core with minimal set of algorithms
(native and GDAL), all other providers should be plugins and installed by
users. Ideally these providers also should be maintained by interested groups
of devs/users, not by QGIS devs.
Post by Victor Olaya
The original idea was to have most providers removed...which included
R, GRASS and SAGA as well. Finally, only certain providers were
removed, mostly those that require dependencies not provided with QGIS
(LiDAR tools, R...and OTB)
If maintaining the plugin is complicated and no work is done on that,
then definitely it is better to keep it as core. But we have to make
sure that the user knows that the provider by itself is useless, and
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
My 2 cents
Post by Paolo Cavallini
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion?
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Alexander Bruy
Paolo Cavallini
2018-01-31 10:28:03 UTC
Permalink
Post by Alexander Bruy
We just simply does not have enough resources to maintain more than
600 algorithms from GRASS and SAGA.
I still think that Processing should be in core with minimal set of algorithms
(native and GDAL), all other providers should be plugins and installed by
users. Ideally these providers also should be maintained by interested groups
of devs/users, not by QGIS devs.
I understand the point all too well; the big risk here is these
providers will fall into oblivion, with a big damage to QGIS as a whole.
Overall, I'm in favour of keeping these as a part of the project,
supported by it.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Nyall Dawson
2018-01-31 23:17:28 UTC
Permalink
Post by Alexander Bruy
Another reason to remove providers is that algorithms descriptions
were not updated and maintained. Even now both SAGA and GRASS
providers are not updated to latest stable versions of the corresponding
software.
We just simply does not have enough resources to maintain more than
600 algorithms from GRASS and SAGA. Situation with OTB, was even
worse, it requires special handling of the parameters.
I still think that Processing should be in core with minimal set of algorithms
(native and GDAL), all other providers should be plugins and installed by
users. Ideally these providers also should be maintained by interested groups
of devs/users, not by QGIS devs.
A big +1 to everything Alex said above, and a strong -1 to
reintroducing any of these providers back to master (and also a +1 to
shifting the SAGA provider to a non-default plugin).

My reasons:

- we have a very strong plugin infrastructure designed EXACTLY for
these use cases. Having these providers as separate plugins, free from
the QGIS release cycle makes a ton of sense. Plugins devs can then
push new versions of their providers whenever new versions of the
underlying libraries are available, and not be constrained by waiting
until the next QGIS release.

- years of experience has PROVEN that we do not have the capacity to
maintain these providers ourselves. Numerous developers have tried,
and just been swamped by the amount of work required to keep the
providers stable while the underlying libraries change. The QGIS team
is running at capacity just keeping QGIS maintained and stable - we
CANNOT take on this extra work.

- regardless of the approach taken, things just don't stay stable for
us. E.g. we tried to make the SAGA provider more stable by only
targeting the SAGA LTR release.... yet the SAGA upstream seem to have
abandoned the LTR, leaving it totally broken on newer build chains. I
cannot get SAGA LTR to run on my development platform (Fedora) as it
constantly segfaults. End result - even if I had the time to maintain
this provider, I would be totally unable to. (For this reason I think
SAGA should also be moved to a separate QGIS plugin, leaving on QGIS,
GDAL and GRASS provided in the default install. GDAL and GRASS
(mostly) are always available wherever QGIS is.).

- While QGIS 3.0 has introduced big changes for processing, these are
likely to be the last big changes processing providers will see for
the future. At least for the next 3-4 years (or lifetime of 3.x) the
API will be fixed. And even following that, I'd envision any breaks
introduced in 4.0 to be much less extreme. So while it's painful for
plugins to update their providers for 3.0, it's a one-time pain. In
contrast, expecting QGIS devs to follow the numerous underlying
library changes and version breaks for multiple 3rd party dependancies
is exhausting, ongoing work. It's much more sensible for someone
intimately familiar with those libraries to maintain a plugin which
closely tracks the library development and follow best-practice API
use for that library.


Nyall
Post by Alexander Bruy
Post by Victor Olaya
The original idea was to have most providers removed...which included
R, GRASS and SAGA as well. Finally, only certain providers were
removed, mostly those that require dependencies not provided with QGIS
(LiDAR tools, R...and OTB)
If maintaining the plugin is complicated and no work is done on that,
then definitely it is better to keep it as core. But we have to make
sure that the user knows that the provider by itself is useless, and
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
My 2 cents
Post by Paolo Cavallini
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion?
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Alexander Bruy
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Victor Olaya
2018-02-01 07:21:50 UTC
Permalink
+1 from me. This is exactly the reasoning behind my original proposal
of removing providers from core. Whether the plugin is easy to
maintain or not, it is better to do it outside. People responsible of
the plugin can release whenever they want, and they can do changes
without having to do PRs to the main repo (so less burden for core
devs)

A plugin with a provider has much more to do with the provider app
than with QGIS, and it makes sense to maintain it as a separate thing,
and, if possible, done by the people that better know the external app
(like the OTB team in the case of OTB)

Cheers
Post by Nyall Dawson
Post by Alexander Bruy
Another reason to remove providers is that algorithms descriptions
were not updated and maintained. Even now both SAGA and GRASS
providers are not updated to latest stable versions of the corresponding
software.
We just simply does not have enough resources to maintain more than
600 algorithms from GRASS and SAGA. Situation with OTB, was even
worse, it requires special handling of the parameters.
I still think that Processing should be in core with minimal set of algorithms
(native and GDAL), all other providers should be plugins and installed by
users. Ideally these providers also should be maintained by interested groups
of devs/users, not by QGIS devs.
A big +1 to everything Alex said above, and a strong -1 to
reintroducing any of these providers back to master (and also a +1 to
shifting the SAGA provider to a non-default plugin).
- we have a very strong plugin infrastructure designed EXACTLY for
these use cases. Having these providers as separate plugins, free from
the QGIS release cycle makes a ton of sense. Plugins devs can then
push new versions of their providers whenever new versions of the
underlying libraries are available, and not be constrained by waiting
until the next QGIS release.
- years of experience has PROVEN that we do not have the capacity to
maintain these providers ourselves. Numerous developers have tried,
and just been swamped by the amount of work required to keep the
providers stable while the underlying libraries change. The QGIS team
is running at capacity just keeping QGIS maintained and stable - we
CANNOT take on this extra work.
- regardless of the approach taken, things just don't stay stable for
us. E.g. we tried to make the SAGA provider more stable by only
targeting the SAGA LTR release.... yet the SAGA upstream seem to have
abandoned the LTR, leaving it totally broken on newer build chains. I
cannot get SAGA LTR to run on my development platform (Fedora) as it
constantly segfaults. End result - even if I had the time to maintain
this provider, I would be totally unable to. (For this reason I think
SAGA should also be moved to a separate QGIS plugin, leaving on QGIS,
GDAL and GRASS provided in the default install. GDAL and GRASS
(mostly) are always available wherever QGIS is.).
- While QGIS 3.0 has introduced big changes for processing, these are
likely to be the last big changes processing providers will see for
the future. At least for the next 3-4 years (or lifetime of 3.x) the
API will be fixed. And even following that, I'd envision any breaks
introduced in 4.0 to be much less extreme. So while it's painful for
plugins to update their providers for 3.0, it's a one-time pain. In
contrast, expecting QGIS devs to follow the numerous underlying
library changes and version breaks for multiple 3rd party dependancies
is exhausting, ongoing work. It's much more sensible for someone
intimately familiar with those libraries to maintain a plugin which
closely tracks the library development and follow best-practice API
use for that library.
Nyall
Post by Alexander Bruy
Post by Victor Olaya
The original idea was to have most providers removed...which included
R, GRASS and SAGA as well. Finally, only certain providers were
removed, mostly those that require dependencies not provided with QGIS
(LiDAR tools, R...and OTB)
If maintaining the plugin is complicated and no work is done on that,
then definitely it is better to keep it as core. But we have to make
sure that the user knows that the provider by itself is useless, and
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
My 2 cents
Post by Paolo Cavallini
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion?
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Alexander Bruy
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Paolo Cavallini
2018-02-01 08:31:07 UTC
Permalink
Post by Victor Olaya
+1 from me. This is exactly the reasoning behind my original proposal
of removing providers from core. Whether the plugin is easy to
maintain or not, it is better to do it outside. People responsible of
the plugin can release whenever they want, and they can do changes
without having to do PRs to the main repo (so less burden for core
devs)
A plugin with a provider has much more to do with the provider app
than with QGIS, and it makes sense to maintain it as a separate thing,
and, if possible, done by the people that better know the external app
(like the OTB team in the case of OTB)
Thanks Alex, Nyall, Victor for thoughts. It indeed makes sense.
What can happen realistically is, IMHO:
* someone will pop up taking the now orphaned providers, as it is
happening now for OTB --> success
* no one will find the resources for this, and QGIS will miss hundreds
of important algs --> failure
Of course this second possibility is scary to all those, that rely on
these algs. I am finding easier since too many years to find resources
for fancy styling and other cartographic tasks than for hard core
analyses, so I tend to be pessimistic, but let's see.
Thanks again.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Rashad Kanavath
2018-02-01 10:01:35 UTC
Permalink
Hello,

Processing plugin allows to integrate other toolboxes. IIUC, this was one
of the features of it.
So when you say integration of so and so toolboxes are total mess, you
might think back.

Nobody had seen new changes to otb algs so all of your comments are on old
version. Why so rush?
Okay, it easy to reject stating the same reason over and over again. I
understand that.
What happens at end, a processing plugin with zero providers?

Now the reason OTB had to maintain list of "supported" version is due the
fact that processing plugin does not allow to group parameters.
So the issue of a provider being total mess not fully related to provider
itself. If 90% of provider algorithm which you use, cannot be even
integrated into processing where will be the actual problem. I see a lot of
good changes in qgis processing and providers can greatly benefit from it
now.

All those people blindly rejecting this proposal should have waited for new
changes.
I mean, I just put a proposal to lower the burden as much as possible. And
all those issues raised by Alex, Nyall and others will be considered in the
new diff.
Once I prepare changes, you can reject it!. You are voting -1ss on old
plugin code something I consider a mess to work with for both teams.

My initial question was only to see if there are other issues than the
maintenance overhead?
By now, I conclude that only issue was maintenance overhead and I proposed
it can be solved.
Post by Rashad Kanavath
Post by Alexander Bruy
Another reason to remove providers is that algorithms descriptions
were not updated and maintained. Even now both SAGA and GRASS
providers are not updated to latest stable versions of the corresponding
software.
We just simply does not have enough resources to maintain more than
600 algorithms from GRASS and SAGA. Situation with OTB, was even
worse, it requires special handling of the parameters.
I still think that Processing should be in core with minimal set of
algorithms
Post by Alexander Bruy
(native and GDAL), all other providers should be plugins and installed by
users. Ideally these providers also should be maintained by interested
groups
Post by Alexander Bruy
of devs/users, not by QGIS devs.
A big +1 to everything Alex said above, and a strong -1 to
reintroducing any of these providers back to master (and also a +1 to
shifting the SAGA provider to a non-default plugin).
- we have a very strong plugin infrastructure designed EXACTLY for
these use cases. Having these providers as separate plugins, free from
the QGIS release cycle makes a ton of sense. Plugins devs can then
push new versions of their providers whenever new versions of the
underlying libraries are available, and not be constrained by waiting
until the next QGIS release.
- years of experience has PROVEN that we do not have the capacity to
maintain these providers ourselves. Numerous developers have tried,
and just been swamped by the amount of work required to keep the
providers stable while the underlying libraries change. The QGIS team
is running at capacity just keeping QGIS maintained and stable - we
CANNOT take on this extra work.
- regardless of the approach taken, things just don't stay stable for
us. E.g. we tried to make the SAGA provider more stable by only
targeting the SAGA LTR release.... yet the SAGA upstream seem to have
abandoned the LTR, leaving it totally broken on newer build chains. I
cannot get SAGA LTR to run on my development platform (Fedora) as it
constantly segfaults. End result - even if I had the time to maintain
this provider, I would be totally unable to. (For this reason I think
SAGA should also be moved to a separate QGIS plugin, leaving on QGIS,
GDAL and GRASS provided in the default install. GDAL and GRASS
(mostly) are always available wherever QGIS is.).
- While QGIS 3.0 has introduced big changes for processing, these are
likely to be the last big changes processing providers will see for
the future. At least for the next 3-4 years (or lifetime of 3.x) the
API will be fixed. And even following that, I'd envision any breaks
introduced in 4.0 to be much less extreme. So while it's painful for
plugins to update their providers for 3.0, it's a one-time pain. In
contrast, expecting QGIS devs to follow the numerous underlying
library changes and version breaks for multiple 3rd party dependancies
is exhausting, ongoing work. It's much more sensible for someone
intimately familiar with those libraries to maintain a plugin which
closely tracks the library development and follow best-practice API
use for that library.
Nyall
Post by Alexander Bruy
Post by Victor Olaya
The original idea was to have most providers removed...which included
R, GRASS and SAGA as well. Finally, only certain providers were
removed, mostly those that require dependencies not provided with QGIS
(LiDAR tools, R...and OTB)
If maintaining the plugin is complicated and no work is done on that,
then definitely it is better to keep it as core. But we have to make
sure that the user knows that the provider by itself is useless, and
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
My 2 cents
Post by Paolo Cavallini
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion?
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Alexander Bruy
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Regards,
Rashad
Nyall Dawson
2018-02-01 23:36:42 UTC
Permalink
Post by Rashad Kanavath
Hello,
Processing plugin allows to integrate other toolboxes. IIUC, this was one of
the features of it.
So when you say integration of so and so toolboxes are total mess, you might
think back.
I think there's a misunderstanding/language barrier at play here -
no-one meant that the code is a mess, just that the end result of the
QGIS team trying to maintain 3rd party providers resulted in an
inferior experience all round - for users, qgis developers AND the
underlying library developers (whom I'm sure don't appreciate being
blamed when a QGIS processing algorithm is mis-using their API).

Yes, one of Processing's big strengths is its ability to integrate
other toolboxes and make 3rd party libraries accessible within the
QGIS environment and tie them together into models. Another one of
QGIS' great strengths is its strong plugin ecosystem, which allows
anyone the ability to create plugins and extend QGIS, and not be tied
into the core QGIS development practices and release schedules. That's
why plugin-based providers are the BEST solution in my opinion.
Post by Rashad Kanavath
Nobody had seen new changes to otb algs so all of your comments are on old
version. Why so rush?
Okay, it easy to reject stating the same reason over and over again. I
understand that.
Just to be clear - no-one is commenting on your code quality or
targeting OTB specifically. It's the question of whether or not
Processing providers which depend on other libraries should be
included in the main install or moved to plugins which we are
debating. So please, don't take any of this as criticism of your code
or (much appreciated) efforts.
Post by Rashad Kanavath
What happens at end, a processing plugin with zero providers?
Far from it. My vision would be:

- core install: only includes the native QGIS providers (c++, Python
and 3D) and the GDAL/OGR provider (since it's impossible to build QGIS
without GDAL we can be 100% sure that it's always available wherever
QGIS is installed). Maybe GRASS provider too, since GRASS is very
heavily linked into QGIS due to the GRASS provider and c++ GRASS
plugin.
- via QGIS' plugin ecosystem: a whole world of providers ready to
install for users, including SAGA, OTB, lastools, R, PySAL, etc...
Post by Rashad Kanavath
Now the reason OTB had to maintain list of "supported" version is due the
fact that processing plugin does not allow to group parameters.
So the issue of a provider being total mess not fully related to provider
itself. If 90% of provider algorithm which you use, cannot be even
integrated into processing where will be the actual problem. I see a lot of
good changes in qgis processing and providers can greatly benefit from it
now.
Can you elaborate on this please? I'm not aware what the "group
parameters" issue is.
Post by Rashad Kanavath
All those people blindly rejecting this proposal should have waited for new
changes.
I mean, I just put a proposal to lower the burden as much as possible. And
all those issues raised by Alex, Nyall and others will be considered in the
new diff.
Once I prepare changes, you can reject it!. You are voting -1ss on old
plugin code something I consider a mess to work with for both teams.
Why? In the past we've always found that discussing plans upfront
benefits everyone and prevents development effort in an approach which
won't be accepted upstream.

In summary, can you please outline the reasons why you think
publishing this as a plugin is not ideal?

Nyall
Rashad Kanavath
2018-02-02 10:47:29 UTC
Permalink
Post by Rashad Kanavath
Post by Rashad Kanavath
Hello,
Processing plugin allows to integrate other toolboxes. IIUC, this was
one of
Post by Rashad Kanavath
the features of it.
So when you say integration of so and so toolboxes are total mess, you
might
Post by Rashad Kanavath
think back.
I think there's a misunderstanding/language barrier at play here -
no-one meant that the code is a mess, just that the end result of the
QGIS team trying to maintain 3rd party providers resulted in an
inferior experience all round - for users, qgis developers AND the
underlying library developers (whom I'm sure don't appreciate being
blamed when a QGIS processing algorithm is mis-using their API).
Can you elaborate on this QGIS processing algorithm mis-using API ?
Post by Rashad Kanavath
Yes, one of Processing's big strengths is its ability to integrate
other toolboxes and make 3rd party libraries accessible within the
QGIS environment and tie them together into models. Another one of
QGIS' great strengths is its strong plugin ecosystem, which allows
anyone the ability to create plugins and extend QGIS, and not be tied
into the core QGIS development practices and release schedules. That's
why plugin-based providers are the BEST solution in my opinion.
Post by Rashad Kanavath
Nobody had seen new changes to otb algs so all of your comments are on
old
Post by Rashad Kanavath
version. Why so rush?
Okay, it easy to reject stating the same reason over and over again. I
understand that.
Just to be clear - no-one is commenting on your code quality or
targeting OTB specifically. It's the question of whether or not
Processing providers which depend on other libraries should be
included in the main install or moved to plugins which we are
debating. So please, don't take any of this as criticism of your code
or (much appreciated) efforts.
I never said you commented on code quality. I said everyone should comment
on new code.
But there are comments and or decisions made on old one codebase even the
thread started with updating and cleaning up stuff.
And I did said that all your comments about that is valid and reasonable.

Hence, I suggested that to see if all your points to keep providers out
still stands with new code. (not done yet)
For SAGA and others, I cannot speak much because I am not involved in it.
so if it feels like I was targeting OTB, this was not against anything.
But I guess saga or even grass is not planning for similar support for qgis
provider like OTB.
And maybe this is the case for R and others. So that make sense to remove
it from plugin.

If there is one plugin provider which has mostly code tied to processing
then it will be easy for QGIS.
Consider this, OTB plugin was moved out of qgis some time ago and after
that came a lot of changes to qgis processing.
change of parameter names, moving code to c++ etc.. Now GRASS, SAGA etc got
all these changes because it was in tree.
I understand the idea behind removing otb. If this provider wasn't such a
mess to maintain it would have stayed. Even though it
attains the same complexity level as others. I don't agree that it should
be so complex.

And IMHO, the point being this providers are hard to maintain itself lies
with processing plugin and not individual providers or qgis core.
Be it OTB, GRASS , SAGA or anything.
Post by Rashad Kanavath
Post by Rashad Kanavath
What happens at end, a processing plugin with zero providers?
- core install: only includes the native QGIS providers (c++, Python
and 3D) and the GDAL/OGR provider (since it's impossible to build QGIS
without GDAL we can be 100% sure that it's always available wherever
QGIS is installed). Maybe GRASS provider too, since GRASS is very
heavily linked into QGIS due to the GRASS provider and c++ GRASS
plugin.
In case of GDAL, I agree. But I (and many other users) don't have GRASS and
SAGA installed.
So this is some kind of assumption that I cannot fully agree. GRASS
provider still has all of descriptor for grass7 in your tree.
And this was/is the case of OTB for now. New changes will not need a
descriptor file in your tree.
OTB installation will provide a descriptor file in the format needed by
qgis processing!
- via QGIS' plugin ecosystem: a whole world of providers ready to
Post by Rashad Kanavath
install for users, including SAGA, OTB, lastools, R, PySAL, etc...
Yes. most of the concern was more with installation of external tools for
users.
From this and other threads, keeping processing providers outside was not
related to amount or quality code or targetted at specific tool.
Just that it requires external tools. We understand that totally. But what
we don't agree is your assumption that so and so tools is already for most
of qgis users.
Even if grass or saga is installed, one has to configure it to use. So the
problem of installing something for users seems to be key here.
Now GDAL is not considered an external tool because you need it to build
qgis. I am talking of other providers and not just OTB.
Post by Rashad Kanavath
Post by Rashad Kanavath
Now the reason OTB had to maintain list of "supported" version is due the
fact that processing plugin does not allow to group parameters.
So the issue of a provider being total mess not fully related to provider
itself. If 90% of provider algorithm which you use, cannot be even
integrated into processing where will be the actual problem. I see a lot
of
Post by Rashad Kanavath
good changes in qgis processing and providers can greatly benefit from it
now.
Can you elaborate on this please? I'm not aware what the "group
parameters" issue is.
OTB application allows use of parameter with sub parameters,
One such example is here:
https://www.orfeo-toolbox.org/CookBook/Applications/app_Smoo
thing.html#parameters
https://www.orfeo-toolbox.org/CookBook/Applications/app_Trai
nImagesClassifier.html#parameters

There is a parameter called "type" in Smoothing application. "type"
parameter has list of values such as mean, gaussian and anidif.
The ui widget of this parameter is obviously a comboBox.
This flexibility allows users to select which type of smoothing to apply.
Rather than having a three application namely
smoothing_mean, smoothing_gaussian, smoothing_anidif

so parameter type.* (type.mean.radius, type.gaussian.radius, etc..) etc are
child of parameter called "type".
So you can call "type" parameter a group.

A user of application can select -type gaussian -type.gaussian.radius 3
when launching application.
otb application check for invalid cases such as if type is gaussian then
one cannot use -type.mean.radius
This is okay for command line, but for graphical interface one has to hide
/show parameters based on the value of it's parent.

This is a limitation of qgis processing that upstream was forced to split
applications based on group. I am not blaming or targeting here.
Just saying about a missing feature for support for one provider.

QGIS users now have three application for smoothing and maybe 10 or more
application for TrainImagesClassifer.
This make users of otb provider as well as developers of both qgis and otb
unhappy. Perimeter of bug report is getting more and more
And nobody wants to work on this part.

Now, this can be fixed by allowing plugin providers to hide/show any of the
parameters in the ui.
The rule to hide/show can be left out to provider itself. This change must
go in qgis processing and not in provider plugin.

If qgis processing plugin has a new widget that allows this, then things go
easy. You can keep providers as external plugin.
But provider will make use a qgs processing ui widget which is already in
tree. That makes sure it is tested properly among other things.
Provider will be free from hacks such as splitting applications and doing
stuff specific to some version(s)
And this widget can be reused by existing or new providers also.

I am attaching some screenshots from otb application viewer to visual
explanation.
scheme_parameter.png shows the grouping of parameters.
and the other two explains the show/hide of parameters based on value of
parameter "type"

screenshots:
Loading Image...
Loading Image...
Loading Image...
Post by Rashad Kanavath
Post by Rashad Kanavath
All those people blindly rejecting this proposal should have waited for
new
Post by Rashad Kanavath
changes.
I mean, I just put a proposal to lower the burden as much as possible.
And
Post by Rashad Kanavath
all those issues raised by Alex, Nyall and others will be considered in
the
Post by Rashad Kanavath
new diff.
Once I prepare changes, you can reject it!. You are voting -1ss on old
plugin code something I consider a mess to work with for both teams.
Why? In the past we've always found that discussing plans upfront
benefits everyone and prevents development effort in an approach which
won't be accepted upstream.
In summary, can you please outline the reasons why you think
publishing this as a plugin is not ideal?
Nyall
--
Regards,
Rashad
Paolo Cavallini
2018-02-02 11:11:56 UTC
Permalink
Hi Rashad,
Post by Rashad Kanavath
A user of application can select -type gaussian -type.gaussian.radius 3
when launching application.
otb application check for invalid cases such as if type is gaussian then
one cannot use -type.mean.radius
This is okay for command line, but for graphical interface one has to
hide /show parameters based on the value of it's parent.
This is a limitation of qgis processing that upstream was forced to
split applications based on group. I am not blaming or targeting here.
Just saying about a missing feature for support for one provider.
Very interesting. Wouldn't make sense to have it upstream? This would be
useful also for other providers.
Post by Rashad Kanavath
 QGIS users now have three application for smoothing and maybe 10 or
more application for TrainImagesClassifer.
This was originally made on purpose, with the idea of having simplified
atomic commands. Nowe the ecosystem is more riche, and there is room for
more complex interfaces.

Thanks for your input.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Pedro Venâncio
2018-02-02 12:40:47 UTC
Permalink
Hi all,

I think this new approach is more or less the same approach used in the
early times of Sextante, before it is ported to QGIS core and become
Processing.

I remember that Sextante at that time had a very good feature, being an
external plugin, that was the updates. They were not dependent of the QGIS
release schedule, and so bugs were solved and new versions made available
very quickly.

Having the providers as plugins, we will regain this flexibility. The
counterpart is that there may be providers that may no longer exist in
Processing, due to lack of workforce. But getting them integrated, without
this workforce, lacking maintenance, is not a good idea either.

So I think this new approach, taking pros and cons, might work well.

I understand Rashad who thinks it would be better to have all providers,
out-of-the-box, after installing QGIS, but we can not get the best of both
worlds in just one. And it seems to me, that OTB's integration into QGIS is
assured, at least as much as it depends on Rashad's will!

Best regards,
Pedro
Post by Paolo Cavallini
Hi Rashad,
Post by Rashad Kanavath
A user of application can select -type gaussian -type.gaussian.radius 3
when launching application.
otb application check for invalid cases such as if type is gaussian then
one cannot use -type.mean.radius
This is okay for command line, but for graphical interface one has to
hide /show parameters based on the value of it's parent.
This is a limitation of qgis processing that upstream was forced to
split applications based on group. I am not blaming or targeting here.
Just saying about a missing feature for support for one provider.
Very interesting. Wouldn't make sense to have it upstream? This would be
useful also for other providers.
Post by Rashad Kanavath
QGIS users now have three application for smoothing and maybe 10 or
more application for TrainImagesClassifer.
This was originally made on purpose, with the idea of having simplified
atomic commands. Nowe the ecosystem is more riche, and there is room for
more complex interfaces.
Thanks for your input.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Rashad Kanavath
2018-01-31 10:23:34 UTC
Permalink
Post by Victor Olaya
The original idea was to have most providers removed...which included
R, GRASS and SAGA as well. Finally, only certain providers were
removed, mostly those that require dependencies not provided with QGIS
(LiDAR tools, R...and OTB)
If maintaining the plugin is complicated and no work is done on that,
then definitely it is better to keep it as core. But we have to make
sure that the user knows that the provider by itself is useless, and
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
Okay.
I propose the following change.
an otb provider without much complication to maintain for qgis and is not
tied to specific otb version
A provider will allow to download && install latest otb binary packages
(optional) for user's who don't have OTB.
Ofcourse, a download of otb isn't just download, it will configure itself
for qgis processing.
So the provider itself is usable in a way without otb, because it can
handle installation if one needs to.
Post by Victor Olaya
My 2 cents
Post by Paolo Cavallini
Post by Rashad Kanavath
Thanks Paolo,
Can you give a link to that discussion?
https://lists.osgeo.org/pipermail/qgis-developer/2017-May/048470.html
Surely Victor and Alex can provide further details.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Regards,
Rashad
Paolo Cavallini
2018-01-31 10:25:18 UTC
Permalink
Post by Victor Olaya
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
OTB is included in the standalone for win (at least for 32 bit, IMHO it
should be also in 64 bit). In Debian this is not an issue.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Alexander Bruy
2018-01-31 10:26:31 UTC
Permalink
It is included, but this version is really outdated isn't it?
Post by Paolo Cavallini
Post by Victor Olaya
that OTB has to be installed separately. Otherwise, we will see
confusion and i think it will not be a good idea
OTB is included in the standalone for win (at least for 32 bit, IMHO it
should be also in 64 bit). In Debian this is not an issue.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Alexander Bruy
Paolo Cavallini
2018-01-31 10:29:17 UTC
Permalink
Post by Alexander Bruy
It is included, but this version is really outdated isn't it?
we can seek additional resources to maintain it. Perhaps OTB team is
interested in maintaining it up to date?
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Rashad Kanavath
2018-01-31 10:43:39 UTC
Permalink
Post by Paolo Cavallini
Post by Alexander Bruy
It is included, but this version is really outdated isn't it?
we can seek additional resources to maintain it. Perhaps OTB team is
interested in maintaining it up to date?
the old plugin code had lot of issue like maintain list of supported
version, keeping a backlist and whilelist for supported applications (don't
ask),
splitting of apps because groups aren't supported to name a few.
And I think this one of the blocking point for qgis and also otb. It need
to put a lot of efforts to maintain otb provider and cost is triple
compared to others
If the plugin code was simply generic, qgis would have done maintain it
like old way. Because you only need to change plugin if there is are API
changes,
or some stuff that is made into processing itself. This one will be easy
because all source is in tree you can find places where it needs to change
like:

-from processing.core.AlgorithmProvider import AlgorithmProvider
+from qgis.core import QgsProcessingProvider

And this are back normal.

What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible.
So qgis can focus on issue related only to that and after a while things
will stabilize.

Indeed, if there is something broken in this algo otb team will help fix it.

All the best.
Post by Paolo Cavallini
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Regards,
Rashad
Paolo Cavallini
2018-02-01 08:23:30 UTC
Permalink
Hi Rashad,
Post by Rashad Kanavath
What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible. 
So qgis can focus on issue related only to that and after a while things
will stabilize.
Indeed, if there is something broken in this algo otb team will help fix it.
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows? It would be
good to make sure we always have the most appropriate version, both for
32 and 64 bit.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Jürgen E. Fischer
2018-02-01 08:32:54 UTC
Permalink
Hi Paolo,
Post by Paolo Cavallini
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows?
The standalone installers are made from OSGeo4W packages. So otb should be
updated there. IIRC Julien Malik did the last update to 5.2 there.


Jürgen
--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de
QGIS release manager (PSC) Germany IRC: jef on FreeNode
Rashad Kanavath
2018-02-01 09:35:40 UTC
Permalink
Post by Jürgen E. Fischer
Hi Paolo,
Post by Paolo Cavallini
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows?
The standalone installers are made from OSGeo4W packages. So otb should be
updated there. IIRC Julien Malik did the last update to 5.2 there.
OTB is not using ogeo4w for a long time. So OSGeo4W users are stuck on old
version. We have zero interest in pushing packages back again
Windows users can use otb from its binary package, build from source , or
maintain osgeo4w or whatever seems interest.
Post by Jürgen E. Fischer
JÃŒrgen
--
JÃŒrgen E. Fischer norBIT GmbH Tel.
+49-4931-918175-31
Dipl.-Inf. (FH) Rheinstraße 13 Fax.
+49-4931-918175-50
Software Engineer D-26506 Norden
http://www.norbit.de
QGIS release manager (PSC) Germany IRC: jef on FreeNode
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Regards,
Rashad
Paolo Cavallini
2018-02-01 10:57:48 UTC
Permalink
Post by Rashad Kanavath
OTB is not using ogeo4w for a long time. So OSGeo4W users are stuck on
old version. We have zero interest in pushing packages back again
Windows users can use otb from its binary package, build from source ,
or maintain osgeo4w or whatever seems interest.
Thanks Rashad for clarifying. In this case, I'd strongly advise removing
OTB from osgeo4w ASAP.
Could you or Julien please do it?
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Rashad Kanavath
2018-02-01 11:01:12 UTC
Permalink
Post by Paolo Cavallini
Post by Rashad Kanavath
OTB is not using ogeo4w for a long time. So OSGeo4W users are stuck on
old version. We have zero interest in pushing packages back again
Windows users can use otb from its binary package, build from source ,
or maintain osgeo4w or whatever seems interest.
Thanks Rashad for clarifying. In this case, I'd strongly advise removing
OTB from osgeo4w ASAP.
Could you or Julien please do it?
I think jef can do this part. I don't have access to repo
Post by Paolo Cavallini
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Regards,
Rashad
Paolo Cavallini
2018-02-01 11:04:58 UTC
Permalink
Post by Rashad Kanavath
I think jef can do this part. I don't have access to repo
OK, thanks. Could you please open a ticket on osgeo4w tracker?
Thanks.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Jürgen E. Fischer
2018-02-01 17:18:47 UTC
Permalink
Hi Paolo,
Post by Paolo Cavallini
Post by Rashad Kanavath
I think jef can do this part. I don't have access to repo
OK, thanks. Could you please open a ticket on osgeo4w tracker?
No need. otb has been marked _obsolete in OSGeo4W.


Jürgen
--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de
Paolo Cavallini
2018-02-02 07:46:21 UTC
Permalink
Post by Jürgen E. Fischer
Hi Paolo,
Post by Paolo Cavallini
Post by Rashad Kanavath
I think jef can do this part. I don't have access to repo
OK, thanks. Could you please open a ticket on osgeo4w tracker?
No need. otb has been marked _obsolete in OSGeo4W.
fine then
thanks
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
G. Allegri
2018-02-01 08:33:54 UTC
Permalink
+1 also from me to the removal of the providers from core. IMHO I would
also remove GRASS, but I understand it could be a tough decision.
The provider system + plugin architecture provide all the means to keep
them separate from the core.

We all agree, I guess, that less is better then more, to guarantee the
highest quality we can afford.

Giovanni
Post by Paolo Cavallini
Hi Rashad,
Post by Rashad Kanavath
What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible.
So qgis can focus on issue related only to that and after a while things
will stabilize.
Indeed, if there is something broken in this algo otb team will help fix
it.
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows? It would be
good to make sure we always have the most appropriate version, both for
32 and 64 bit.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Werner Macho
2018-02-01 08:39:06 UTC
Permalink
Hi all,

I am also in favour of removing all the work that is not directly related
to QGIS.
But I'd also like to ask if there is already a plugin available (on
github?) that for e.g. readds the GRASS functionality?
Furthermore I'd like to know if it is possible to create a plugin for
official stable GRASS and GRASS development version, which in the best case
can be installed side by side?
(Same applies to SAGA and all the other providers).

I am sure there is a huge possibility that an ecosystem will be created
around those kind of plugins.

kind regards
Werner
Post by G. Allegri
+1 also from me to the removal of the providers from core. IMHO I would
also remove GRASS, but I understand it could be a tough decision.
The provider system + plugin architecture provide all the means to keep
them separate from the core.
We all agree, I guess, that less is better then more, to guarantee the
highest quality we can afford.
Giovanni
Post by Paolo Cavallini
Hi Rashad,
Post by Rashad Kanavath
What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible.
So qgis can focus on issue related only to that and after a while things
will stabilize.
Indeed, if there is something broken in this algo otb team will help
fix it.
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows? It would be
good to make sure we always have the most appropriate version, both for
32 and 64 bit.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Alexander Bruy
2018-02-01 10:00:13 UTC
Permalink
Hi Werner,

there are no plugins for SAGA and GRASS, but this is just because they
are in core.
Also seems we need to clarify, removal from core does not means wiping
all stuff,
this just mean that existing code will be separated into plugin and
published on GitHub
and QGIS plugins repository. Then interested people can take ownership
and continue
development and maintenance.

Exactly in the same way we removed LiDAR tools provider (now
maintained by Martin Isenburg)
and TauDEM provider (maintained by me, along with several others
Processing providers).
Post by Werner Macho
Hi all,
I am also in favour of removing all the work that is not directly related to
QGIS.
But I'd also like to ask if there is already a plugin available (on github?)
that for e.g. readds the GRASS functionality?
Furthermore I'd like to know if it is possible to create a plugin for
official stable GRASS and GRASS development version, which in the best case
can be installed side by side?
(Same applies to SAGA and all the other providers).
I am sure there is a huge possibility that an ecosystem will be created
around those kind of plugins.
kind regards
Werner
Post by G. Allegri
+1 also from me to the removal of the providers from core. IMHO I would
also remove GRASS, but I understand it could be a tough decision.
The provider system + plugin architecture provide all the means to keep
them separate from the core.
We all agree, I guess, that less is better then more, to guarantee the
highest quality we can afford.
Giovanni
Post by Paolo Cavallini
Hi Rashad,
Post by Rashad Kanavath
What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible.
So qgis can focus on issue related only to that and after a while things
will stabilize.
Indeed, if there is something broken in this algo otb team will help fix it.
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows? It would be
good to make sure we always have the most appropriate version, both for
32 and 64 bit.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alexander Bruy
Werner Macho
2018-02-01 10:07:16 UTC
Permalink
Hi Alex,

Thats exactly what I meant.
I know that (currently) there are no plugin for SAGA and GRASS but if I
understood everything correctly the point is to create such plugins.
My Question was more or less if there is already code existing for it or if
we have to wait until they are removed before the plugins can appear - but
this is now answered.

Waiting for the plugins ;)
And hopefully good documentation howto bring new algorithms alive..

thanks and kind regards
Werner
Post by Alexander Bruy
Hi Werner,
there are no plugins for SAGA and GRASS, but this is just because they
are in core.
Also seems we need to clarify, removal from core does not means wiping
all stuff,
this just mean that existing code will be separated into plugin and
published on GitHub
and QGIS plugins repository. Then interested people can take ownership
and continue
development and maintenance.
Exactly in the same way we removed LiDAR tools provider (now
maintained by Martin Isenburg)
and TauDEM provider (maintained by me, along with several others
Processing providers).
Post by Werner Macho
Hi all,
I am also in favour of removing all the work that is not directly
related to
Post by Werner Macho
QGIS.
But I'd also like to ask if there is already a plugin available (on
github?)
Post by Werner Macho
that for e.g. readds the GRASS functionality?
Furthermore I'd like to know if it is possible to create a plugin for
official stable GRASS and GRASS development version, which in the best
case
Post by Werner Macho
can be installed side by side?
(Same applies to SAGA and all the other providers).
I am sure there is a huge possibility that an ecosystem will be created
around those kind of plugins.
kind regards
Werner
Post by G. Allegri
+1 also from me to the removal of the providers from core. IMHO I would
also remove GRASS, but I understand it could be a tough decision.
The provider system + plugin architecture provide all the means to keep
them separate from the core.
We all agree, I guess, that less is better then more, to guarantee the
highest quality we can afford.
Giovanni
Post by Paolo Cavallini
Hi Rashad,
Post by Rashad Kanavath
What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible.
So qgis can focus on issue related only to that and after a while things
will stabilize.
Indeed, if there is something broken in this algo otb team will help fix it.
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows? It would be
good to make sure we always have the most appropriate version, both for
32 and 64 bit.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alexander Bruy
Alexander Bruy
2018-02-01 10:16:49 UTC
Permalink
Hi Werner,

code already here:
- grass https://github.com/qgis/QGIS/tree/master/python/plugins/processing/algs/grass7
- saga https://github.com/qgis/QGIS/tree/master/python/plugins/processing/algs/saga

Only small updates are needed to turn it into plugins. I can turn them
into plugins and
publish if there is an agreement about this approach. Then I will be
happy to transfer
ownership to any interested person.

Some documentation about adding new algorithms to GRASS provider can be found
in https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/grass7/grass7.txt,
but it is slightly outdated. Same approach used for SAGA.
Post by Werner Macho
Hi Alex,
Thats exactly what I meant.
I know that (currently) there are no plugin for SAGA and GRASS but if I
understood everything correctly the point is to create such plugins.
My Question was more or less if there is already code existing for it or if
we have to wait until they are removed before the plugins can appear - but
this is now answered.
Waiting for the plugins ;)
And hopefully good documentation howto bring new algorithms alive..
thanks and kind regards
Werner
Post by Alexander Bruy
Hi Werner,
there are no plugins for SAGA and GRASS, but this is just because they
are in core.
Also seems we need to clarify, removal from core does not means wiping
all stuff,
this just mean that existing code will be separated into plugin and
published on GitHub
and QGIS plugins repository. Then interested people can take ownership
and continue
development and maintenance.
Exactly in the same way we removed LiDAR tools provider (now
maintained by Martin Isenburg)
and TauDEM provider (maintained by me, along with several others
Processing providers).
Post by Werner Macho
Hi all,
I am also in favour of removing all the work that is not directly related to
QGIS.
But I'd also like to ask if there is already a plugin available (on github?)
that for e.g. readds the GRASS functionality?
Furthermore I'd like to know if it is possible to create a plugin for
official stable GRASS and GRASS development version, which in the best case
can be installed side by side?
(Same applies to SAGA and all the other providers).
I am sure there is a huge possibility that an ecosystem will be created
around those kind of plugins.
kind regards
Werner
Post by G. Allegri
+1 also from me to the removal of the providers from core. IMHO I would
also remove GRASS, but I understand it could be a tough decision.
The provider system + plugin architecture provide all the means to keep
them separate from the core.
We all agree, I guess, that less is better then more, to guarantee the
highest quality we can afford.
Giovanni
Post by Paolo Cavallini
Hi Rashad,
Post by Rashad Kanavath
What I can propose is to lower the burden on maintenance but the code
should be put back to qgis if possible.
So qgis can focus on issue related only to that and after a while things
will stabilize.
Indeed, if there is something broken in this algo otb team will help fix it.
thanks for your offer, much appreciated. Would this include also the
inclusion of OTB in the standalone installers for Windows? It would be
good to make sure we always have the most appropriate version, both for
32 and 64 bit.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alexander Bruy
--
Alexander Bruy
Paolo Cavallini
2018-02-01 11:01:10 UTC
Permalink
Post by Alexander Bruy
- grass https://github.com/qgis/QGIS/tree/master/python/plugins/processing/algs/grass7
- saga https://github.com/qgis/QGIS/tree/master/python/plugins/processing/algs/saga
Only small updates are needed to turn it into plugins. I can turn them
into plugins and
publish if there is an agreement about this approach. Then I will be
happy to transfer
ownership to any interested person.
Some documentation about adding new algorithms to GRASS provider can be found
in https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/grass7/grass7.txt,
but it is slightly outdated. Same approach used for SAGA.
I agree that, in case we decide to follow this route, publishing a first
version of the providers and having a reasonably complete documentation
on how to proceed is crucial for the future of the providers.
All the best, and of course thanks Alex and others for your work on this!
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Paolo Cavallini
2018-02-01 10:59:03 UTC
Permalink
Post by Alexander Bruy
Exactly in the same way we removed LiDAR tools provider (now
maintained by Martin Isenburg)
and TauDEM provider (maintained by me, along with several others
Processing providers).
and also R provider, which if I understand correctly is orphaned now.
all the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Alexander Bruy
2018-02-01 11:37:53 UTC
Permalink
Post by Paolo Cavallini
and also R provider, which if I understand correctly is orphaned now.
all the best.
Well, to be precise it did not received much attention from its inclusion into
core, there were tickets filed for this Provider which was not addressed for
years, e.g. issues with R plots, handling different types of parameters.

I'm not a bit R fan and user, so updating this provider to QGIS 3 for me is
lower priority then updating other plugins or doing some work on QGIS core.
--
Alexander Bruy
Paolo Cavallini
2018-02-01 11:40:35 UTC
Permalink
Post by Alexander Bruy
I'm not a bit R fan and user, so updating this provider to QGIS 3 for me is
lower priority then updating other plugins or doing some work on QGIS core.
quite understandable, nobody could possibly blame you.
the fact remains that before removal users were able to do stuff in R
from QGIS, and now they can't.
IMHO we should avoid the same for GRASS and SAGA.
All the best, and thanks.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Alexander Bruy
2018-02-01 11:51:23 UTC
Permalink
IMHO, if it is really important for them, they should find a way
to contact devs and support development.

R provider was removed from core in May 2017 and as far as I can see
nobody asked about it in mailing lists. So I suppose it is not important.
Post by Paolo Cavallini
Post by Alexander Bruy
I'm not a bit R fan and user, so updating this provider to QGIS 3 for me is
lower priority then updating other plugins or doing some work on QGIS core.
quite understandable, nobody could possibly blame you.
the fact remains that before removal users were able to do stuff in R
from QGIS, and now they can't.
IMHO we should avoid the same for GRASS and SAGA.
All the best, and thanks.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
--
Alexander Bruy
Paolo Cavallini
2018-02-01 11:57:28 UTC
Permalink
Post by Alexander Bruy
IMHO, if it is really important for them, they should find a way
to contact devs and support development.
R provider was removed from core in May 2017 and as far as I can see
nobody asked about it in mailing lists. So I suppose it is not important.
unfortunately users are often silent. but I agree, this is a way of
stimulating they reactions.
perhaps we could accompany this with a crewdfunding campaing, or anyway
with a wider announcement "if you want it, do something for it".
Thanks.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
matteo
2018-02-03 14:48:33 UTC
Permalink
Hi all,

I followed silently all the thread.

I know all the difficulties and efforts to maintain different providers
in QGIS, even "just" GRASS and SAGA.

Having a so big number of 3 part algs not QGIS natives is, IMHO, one of
the strength of QGIS. Many users tells that they can run all this
algorithms by only learn a single GUI.

I think removing them from core (even if I PERFECTLY understand devs
options) could have a negative impact for all the users outside.

What also about all the beautiful tests (GRASS still there I think and
SAGA temporary turned off because something of Travis)?

Could it be a chance to leave them within QGIS by extending the number
of tests?

Also all the translation works done until know could be lost?


As I said, just my 2cents

and of course, thanks for all the super work and effort put into this issue.

Cheers

Matteo
Jürgen E. Fischer
2018-02-03 16:02:08 UTC
Permalink
Hi Paolo,
Post by Paolo Cavallini
IMHO, if it is really important for them, they should find a way to contact
devs and support development.
R provider was removed from core in May 2017 and as far as I can see nobody
asked about it in mailing lists. So I suppose it is not important.
unfortunately users are often silent. but I agree, this is a way of
stimulating they reactions.
I guess they'll start complaining once the find that GRASS and SAGA have been
removed from the windows standalone. Before hardly anyone will notice - just
like nobody noticed that there were more hidden providers, because their
dependencies were not installed and in turn nobody missed them, when they were
removed again.

The question is who is the driving force behind the provider plugins. If we
want those algorithms in QGIS, we will probably have to maintain the plugins,
if we don't want them to die.

Otherwise if we don't care and just want to enable others to have QGIS
intgration, they'll have to adopt the plugins. That might work better if there
is real interest. But I think they usally prefer their tools to be used in
their own environment and don't care that much about whether it works in QGIS
or not. Is there solid interest of the SAGA or GRASS team to adopt the
providers? Otherwise I guess they'll sooner or later will die.

At the very least the packaging in OSGeo4W will have to adapted. The easiest
way would just to remove the dependencies. This should also kill the current
problem with the 2GB NSIS limit (GRASS depends on python2, SAGA has wxWidgets,
OTB Qt4).

The plugins would be downloaded from within QGIS and instruct the user how
install the rest of the binaries (eg. from OSGeo4W or other sites (like OTB)).

There could also be processing provider plugin packages in OSGeo4W that have
the providers and depend on available binaries there. Although those packages
would need to be available for all or a selection of qgis packages (qgis,
qgis-rel-dev, qgis-ltr, qgis-ltr-dev and qgis-dev).


Jürgen
--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de
Rashad Kanavath
2018-02-04 12:27:37 UTC
Permalink
Post by Jürgen E. Fischer
Hi Paolo,
Post by Paolo Cavallini
IMHO, if it is really important for them, they should find a way to
contact
Post by Paolo Cavallini
devs and support development.
R provider was removed from core in May 2017 and as far as I can see
nobody
Post by Paolo Cavallini
asked about it in mailing lists. So I suppose it is not important.
unfortunately users are often silent. but I agree, this is a way of
stimulating they reactions.
I guess they'll start complaining once the find that GRASS and SAGA have been
removed from the windows standalone. Before hardly anyone will notice - just
like nobody noticed that there were more hidden providers, because their
dependencies were not installed and in turn nobody missed them, when they were
removed again.
The question is who is the driving force behind the provider plugins. If we
want those algorithms in QGIS, we will probably have to maintain the plugins,
if we don't want them to die.
Otherwise if we don't care and just want to enable others to have QGIS
intgration, they'll have to adopt the plugins. That might work better if there
is real interest. But I think they usally prefer their tools to be used in
their own environment and don't care that much about whether it works in QGIS
or not. Is there solid interest of the SAGA or GRASS team to adopt the
providers? Otherwise I guess they'll sooner or later will die.
At the very least the packaging in OSGeo4W will have to adapted. The easiest
way would just to remove the dependencies. This should also kill the current
problem with the 2GB NSIS limit (GRASS depends on python2, SAGA has wxWidgets,
OTB Qt4).
Well, OTB provider plugin will be able to fetch and install otb binaries.
So users installing plugin is the extra step needed.
1. Install QGIS
2. install otb provider plugin
3. select/download && install otb package

If my proposal to keep otb provider in qgis was allowed, then it will be
two simple steps. But I don't see that happening soon.

Anyways, if the provider is a plugin or not, it is important to have api to
show/hide widgets from algorithm dialog.( see my other mail on parameter
groups)
Post by Jürgen E. Fischer
The plugins would be downloaded from within QGIS and instruct the user how
install the rest of the binaries (eg. from OSGeo4W or other sites (like OTB)).
There could also be processing provider plugin packages in OSGeo4W that have
the providers and depend on available binaries there. Although those packages
would need to be available for all or a selection of qgis packages (qgis,
qgis-rel-dev, qgis-ltr, qgis-ltr-dev and qgis-dev).
JÃŒrgen
--
JÃŒrgen E. Fischer norBIT GmbH Tel.
+49-4931-918175-31
Dipl.-Inf. (FH) Rheinstraße 13 Fax.
+49-4931-918175-50
Software Engineer D-26506 Norden
http://www.norbit.de
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Regards,
Rashad
Nyall Dawson
2018-02-05 03:43:43 UTC
Permalink
Well, OTB provider plugin will be able to fetch and install otb binaries. So
users installing plugin is the extra step needed.
1. Install QGIS
2. install otb provider plugin
3. select/download && install otb package
This sounds great - and all the more reason why (in my opinion)
publishing the provider as a separate plugin is appropriate. A lot of
users will only have to make a couple of clicks and have a fully
functional OTB install and processing provider ready to go.

On the other hand, I don't think this approach is suitable at all for
a core provider. What would you propose to do for Linux users? OTB may
or may not be available in their distro's repos (e.g. it's not
available for Fedora), so how would the plugin install the dependency
in this case? Or what about for Windows users who do not have
administrative rights to install software?

I personally don't think there's any way to guarantee that OTB (or
SAGA for that matter) is available for all QGIS installs, even if we
can manually trigger a download and install via a plugin. And if they
aren't, then we make things harder for our users, QGIS trainers and
support providers -- the feature set of a standard QGIS install will
vary greatly depending on the platform it's installed upon and user's
privileges on that platform.

Nyall
Alexander Bruy
2018-02-05 06:31:08 UTC
Permalink
BTW, there is also another possible issue to consider. Where users
will report tickets related to OTB algorithms and who will address
them?
Post by Nyall Dawson
Well, OTB provider plugin will be able to fetch and install otb binaries. So
users installing plugin is the extra step needed.
1. Install QGIS
2. install otb provider plugin
3. select/download && install otb package
This sounds great - and all the more reason why (in my opinion)
publishing the provider as a separate plugin is appropriate. A lot of
users will only have to make a couple of clicks and have a fully
functional OTB install and processing provider ready to go.
On the other hand, I don't think this approach is suitable at all for
a core provider. What would you propose to do for Linux users? OTB may
or may not be available in their distro's repos (e.g. it's not
available for Fedora), so how would the plugin install the dependency
in this case? Or what about for Windows users who do not have
administrative rights to install software?
I personally don't think there's any way to guarantee that OTB (or
SAGA for that matter) is available for all QGIS installs, even if we
can manually trigger a download and install via a plugin. And if they
aren't, then we make things harder for our users, QGIS trainers and
support providers -- the feature set of a standard QGIS install will
vary greatly depending on the platform it's installed upon and user's
privileges on that platform.
Nyall
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alexander Bruy
Paolo Cavallini
2018-02-05 08:23:15 UTC
Permalink
Hi all,
sorry, I miss a point here: we have a dev who volunteers to maintain a
piece of code. He is quite credible. His proposal minimize the overhead
to QGIS code, and moves much of the complexity to an external sw.
Why do we suggest to keep him out of the door?
All the best.
Post by Alexander Bruy
BTW, there is also another possible issue to consider. Where users
will report tickets related to OTB algorithms and who will address
them?
Post by Nyall Dawson
Well, OTB provider plugin will be able to fetch and install otb binaries. So
users installing plugin is the extra step needed.
1. Install QGIS
2. install otb provider plugin
3. select/download && install otb package
This sounds great - and all the more reason why (in my opinion)
publishing the provider as a separate plugin is appropriate. A lot of
users will only have to make a couple of clicks and have a fully
functional OTB install and processing provider ready to go.
On the other hand, I don't think this approach is suitable at all for
a core provider. What would you propose to do for Linux users? OTB may
or may not be available in their distro's repos (e.g. it's not
available for Fedora), so how would the plugin install the dependency
in this case? Or what about for Windows users who do not have
administrative rights to install software?
I personally don't think there's any way to guarantee that OTB (or
SAGA for that matter) is available for all QGIS installs, even if we
can manually trigger a download and install via a plugin. And if they
aren't, then we make things harder for our users, QGIS trainers and
support providers -- the feature set of a standard QGIS install will
vary greatly depending on the platform it's installed upon and user's
privileges on that platform.
Nyall
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Rashad Kanavath
2018-02-05 08:23:36 UTC
Permalink
Post by Alexander Bruy
BTW, there is also another possible issue to consider. Where users
will report tickets related to OTB algorithms and who will address
them?
Come on!. Users already reports issue to OTB if that is the problem.
Do you know any bugs fixed in OTB processing by QGIS team?
To be fair, the processing plugin is in qgis. And no matter how you
explain some users doesn't listen.
Are you saying this as a problem or something plugin would solve?
Post by Alexander Bruy
Post by Nyall Dawson
Post by Rashad Kanavath
Well, OTB provider plugin will be able to fetch and install otb
binaries. So
Post by Nyall Dawson
Post by Rashad Kanavath
users installing plugin is the extra step needed.
1. Install QGIS
2. install otb provider plugin
3. select/download && install otb package
This sounds great - and all the more reason why (in my opinion)
publishing the provider as a separate plugin is appropriate. A lot of
users will only have to make a couple of clicks and have a fully
functional OTB install and processing provider ready to go.
On the other hand, I don't think this approach is suitable at all for
a core provider. What would you propose to do for Linux users? OTB may
or may not be available in their distro's repos (e.g. it's not
available for Fedora), so how would the plugin install the dependency
in this case? Or what about for Windows users who do not have
administrative rights to install software?
I personally don't think there's any way to guarantee that OTB (or
SAGA for that matter) is available for all QGIS installs, even if we
can manually trigger a download and install via a plugin. And if they
aren't, then we make things harder for our users, QGIS trainers and
support providers -- the feature set of a standard QGIS install will
vary greatly depending on the platform it's installed upon and user's
privileges on that platform.
Nyall
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alexander Bruy
--
Regards,
Rashad
Rashad Kanavath
2018-02-05 08:19:06 UTC
Permalink
Post by Rashad Kanavath
Well, OTB provider plugin will be able to fetch and install otb
binaries. So
Post by Rashad Kanavath
users installing plugin is the extra step needed.
1. Install QGIS
2. install otb provider plugin
3. select/download && install otb package
This sounds great - and all the more reason why (in my opinion)
publishing the provider as a separate plugin is appropriate. A lot of
users will only have to make a couple of clicks and have a fully
functional OTB install and processing provider ready to go.
On the other hand, I don't think this approach is suitable at all for
a core provider. What would you propose to do for Linux users? OTB may
or may not be available in their distro's repos (e.g. it's not
available for Fedora), so how would the plugin install the dependency
in this case? Or what about for Windows users who do not have
administrative rights to install software?
did you checked this package?
https://www.orfeo-toolbox.org//packages/OTB-6.4.0-Linux64.run
and others?
https://www.orfeo-toolbox.org//packages/

We don't need admin rights to install otb. just unzip and it works. Same
for mac and linux.
The dependency is libc and libc++ runtime. And I think any centos6 is our
reference platforms.

Issues you asked is why we introduced binary packages for OTB.
I personally don't think there's any way to guarantee that OTB (or
SAGA for that matter) is available for all QGIS installs, even if we
can manually trigger a download and install via a plugin. And if they
aren't, then we make things harder for our users, QGIS trainers and
support providers -- the feature set of a standard QGIS install will
vary greatly depending on the platform it's installed upon and user's
privileges on that platform.
Nyall
--
Regards,
Rashad
Paolo Cavallini
2018-02-05 08:28:21 UTC
Permalink
Post by Rashad Kanavath
If my proposal to keep otb provider in qgis was allowed, then it will be
two simple steps. But I don't see that happening soon.
nothing is decided yet.
Post by Rashad Kanavath
Anyways, if the provider is a plugin or not, it is important to have api
to show/hide widgets from algorithm dialog.( see my other mail on
parameter groups)
I believe this would be useful for other backends too - what is your
opinion?

All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Paolo Cavallini
2018-02-05 08:26:54 UTC
Permalink
Hi Juergen,
Post by Jürgen E. Fischer
The question is who is the driving force behind the provider plugins. If we
want those algorithms in QGIS, we will probably have to maintain the plugins,
if we don't want them to die.
Otherwise if we don't care and just want to enable others to have QGIS
intgration, they'll have to adopt the plugins. That might work better if there
is real interest. But I think they usally prefer their tools to be used in
their own environment and don't care that much about whether it works in QGIS
or not. Is there solid interest of the SAGA or GRASS team to adopt the
providers? Otherwise I guess they'll sooner or later will die.
agreed fully
Post by Jürgen E. Fischer
At the very least the packaging in OSGeo4W will have to adapted. The easiest
way would just to remove the dependencies. This should also kill the current
problem with the 2GB NSIS limit (GRASS depends on python2, SAGA has wxWidgets,
OTB Qt4).
The plugins would be downloaded from within QGIS and instruct the user how
install the rest of the binaries (eg. from OSGeo4W or other sites (like OTB)).
IMHO anything that does not work out of the box will be just
unaccessible for a large part of users. Standalone packages have been a
key of success for QGIS.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Helmut Kudrnovsky
2018-02-05 11:20:57 UTC
Permalink
Post by Jürgen E. Fischer
Otherwise if we don't care and just want to enable others to have QGIS
intgration, they'll have to adopt the plugins. That might work better if there
is real interest. But I think they usally prefer their tools to be used in
their own environment and don't care that much about whether it works in QGIS
<or not. Is there solid interest of the SAGA or GRASS team to adopt the
Post by Jürgen E. Fischer
providers? Otherwise I guess they'll sooner or later will die.
quickly screened the GRASS MLs, I can't find any entry that the GRASS
community was ever asked if there could be e.g. a shared attempt for an
automatization to create/maintain the plugin code.

Looking at the new OSGeo website:

*Desktop Applications*

-QGIS Desktop
-GRASS GIS

*Geospatial Libraries*

-Orfeo ToolBox
-GDAL/OGR

*Meta CRS Initiative*

-PROJ4

Most of the software mentioned in this thread are projects under the common
umbrella of OSGeo.

An option may be to ask that OSGeo plays a more proactive role in helping to
coordinate and supporting (technically/financally/...) such inter-project
challenges.

I will forward a short summary of this thread to the GRASS community.



-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
Rashad Kanavath
2018-02-05 18:00:07 UTC
Permalink
Hello all,

Here is a PR to allow integration of otb smoothly. tests are all passing,
commits are squashed and ready to review.
https://github.com/qgis/QGIS/pull/6272
I did a small fix to control visibility of parameter from provider.
Addition of specific parameter class for OTB is not included in this PR.

Thanks for feedback
Post by Helmut Kudrnovsky
Post by Jürgen E. Fischer
Otherwise if we don't care and just want to enable others to have QGIS
intgration, they'll have to adopt the plugins. That might work better if
there
Post by Jürgen E. Fischer
is real interest. But I think they usally prefer their tools to be used
in
Post by Jürgen E. Fischer
their own environment and don't care that much about whether it works in
QGIS
<or not. Is there solid interest of the SAGA or GRASS team to adopt the
Post by Jürgen E. Fischer
providers? Otherwise I guess they'll sooner or later will die.
quickly screened the GRASS MLs, I can't find any entry that the GRASS
community was ever asked if there could be e.g. a shared attempt for an
automatization to create/maintain the plugin code.
*Desktop Applications*
-QGIS Desktop
-GRASS GIS
*Geospatial Libraries*
-Orfeo ToolBox
-GDAL/OGR
*Meta CRS Initiative*
-PROJ4
Most of the software mentioned in this thread are projects under the common
umbrella of OSGeo.
An option may be to ask that OSGeo plays a more proactive role in helping to
coordinate and supporting (technically/financally/...) such inter-project
challenges.
I will forward a short summary of this thread to the GRASS community.
-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-
f4099106.html
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Regards,
Rashad
Paolo Cavallini
2018-02-05 19:00:19 UTC
Permalink
Post by Rashad Kanavath
Hello all,
Here is a PR to allow integration of otb smoothly. tests are all
passing, commits are squashed and ready to review.
https://github.com/qgis/QGIS/pull/6272
I did a small fix to control visibility of parameter from provider.
Addition of specific parameter class for OTB is not included in this PR.
+1 for the general idea, pending a code review.
Thanks Rashad.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Helmut Kudrnovsky
2018-02-05 21:03:01 UTC
Permalink
Post by Rashad Kanavath
Hello all,
Here is a PR to allow integration of otb smoothly. tests are all passing,
commits are squashed and ready to review.
https://github.com/qgis/QGIS/pull/6272
I did a small fix to control visibility of parameter from provider.
Addition of specific parameter class for OTB is not included in this PR.
Thanks for feedback
On Mon, Feb 5, 2018 at 12:20 PM, Helmut Kudrnovsky &lt;
Post by Jürgen E. Fischer
Otherwise if we don't care and just want to enable others to have QGIS
intgration, they'll have to adopt the plugins. That might work better
if
there
Post by Jürgen E. Fischer
is real interest. But I think they usally prefer their tools to be used
in
Post by Jürgen E. Fischer
their own environment and don't care that much about whether it works in
QGIS
&lt;or not. Is there solid interest of the SAGA or GRASS team to adopt
the
&gt; >providers? Otherwise I guess they'll sooner or later will die.
quickly screened the GRASS MLs, I can't find any entry that the GRASS
community was ever asked if there could be e.g. a shared attempt for an
automatization to create/maintain the plugin code.
*Desktop Applications*
-QGIS Desktop
-GRASS GIS
*Geospatial Libraries*
-Orfeo ToolBox
-GDAL/OGR
*Meta CRS Initiative*
-PROJ4
Most of the software mentioned in this thread are projects under the common
umbrella of OSGeo.
An option may be to ask that OSGeo plays a more proactive role in helping to
coordinate and supporting (technically/financally/...) such inter-project
challenges.
I will forward a short summary of this thread to the GRASS community.
and see here for a follow up in the GRASS community:

http://osgeo-org.1560.x6.nabble.com/Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352828.html
http://osgeo-org.1560.x6.nabble.com/Re-GRASS-user-Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352836.html




-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
Paolo Cavallini
2018-02-06 16:44:18 UTC
Permalink
Post by Helmut Kudrnovsky
http://osgeo-org.1560.x6.nabble.com/Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352828.html
http://osgeo-org.1560.x6.nabble.com/Re-GRASS-user-Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352836.html
Thank you Helmut. Following our PSC meeting, I'll start an open
discussion with all parties involved. Looking forward for your input.
I assumed GRASS-dev was the list to contact, but you sent it on
GRASS-users, any special reason for this?
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Helmut Kudrnovsky
2018-02-06 23:13:47 UTC
Permalink
Hi Paolo,
Post by Paolo Cavallini
Post by Helmut Kudrnovsky
http://osgeo-org.1560.x6.nabble.com/Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352828.html
http://osgeo-org.1560.x6.nabble.com/Re-GRASS-user-Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352836.html
Thank you Helmut. Following our PSC meeting, I'll start an open
discussion with all parties involved. Looking forward for your input.
I assumed GRASS-dev was the list to contact, but you sent it on
GRASS-users, any special reason for this?
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
I sent it to GRASS users as it seems to me to be more a community issue than
a dev issue.

Thinking about/hearing to your user base helps you to improve decisions on
the dev side.

see

https://trac.osgeo.org/grass/wiki/GSoC/2018#ImproveGRASSintegrationinQGIS3

as a starting point to a possible closer OSGeo inter-project communication
and collaboration.

Being an OSGeo GSoC/GCI admin now for some time, such opportunities or
similar to improve inter-project collaboration seem to be really under-used
at the moment.

I'll start a new thread about the GSoC idea to attract new interested
people.






-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
Helmut Kudrnovsky
2018-02-06 23:47:11 UTC
Permalink
Post by Paolo Cavallini
Post by Helmut Kudrnovsky
http://osgeo-org.1560.x6.nabble.com/Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352828.html
http://osgeo-org.1560.x6.nabble.com/Re-GRASS-user-Keeping-GRASS-OTB-algorithm-in-qgis-processing-td5352836.html
Thank you Helmut. Following our PSC meeting, I'll start an open
discussion with all parties involved. Looking forward for your input.
I assumed GRASS-dev was the list to contact, but you sent it on
GRASS-users, any special reason for this?
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
And on the dev side, some ideas and discussions are floating around:

e.g.

https://lists.osgeo.org/pipermail/grass-dev/2018-February/087388.html



-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
Giovanni Manghi
2018-02-01 11:22:28 UTC
Permalink
Hi all,
Post by Alexander Bruy
Only small updates are needed to turn it into plugins. I can turn them
into plugins and
publish if there is an agreement about this approach. Then I will be
happy to transfer
ownership to any interested person.
I could take care of the description files of such plugins (as I do it
in the core providers), but... I'm not really sure about removing
GRASS and SAGA from QGIS installers. Is true that most if users don't
do any analysis, but having hundreds of tools (that we know for sure
will never be implemented as native QGIS tools) out of the box was and
still is a very important "selling" point. I ask to please take this
decision keeping this aspect in mind. Regarding SAGA LTR I understand
the frustration pf many Linux users (I'm among them), but we have to
look also at the big picture, and that is Windows. After having
decided to support only SAGA LTR (because we know we can't keep the
pace with non LTR SAGA underlying changes) we are now finally at a
point (at least in QGIS 2.18) where users can reliably count with this
tools.

cheers!
Alexander Bruy
2018-02-01 11:28:36 UTC
Permalink
After having decided to support only SAGA LTR (because we know we can't keep the
pace with non LTR SAGA underlying changes) we are now finally at a
point (at least in QGIS 2.18) where users can reliably count with this
tools.
I'm afraid it is not true, there are lot of issue even with LTR SAGA, see
for example https://issues.qgis.org/issues/17726. There are also some
recent threads in developers mailing list about broked SAGA and GRASS
modules.
--
Alexander Bruy
Giovanni Manghi
2018-02-01 12:08:52 UTC
Permalink
Hi Alex,
Post by Alexander Bruy
I'm afraid it is not true, there are lot of issue even with LTR SAGA, see
for example https://issues.qgis.org/issues/17726. There are also some
recent threads in developers mailing list about broked SAGA and GRASS
modules.
ok is not perfect :) but is for sure is much better compared when we
tried to keep the pace with SAGA changes.

-- G --
Alexander Bruy
2018-02-01 12:13:55 UTC
Permalink
Hi Giovanni,

my point was that we still can not maintain hundreds of algorithms.
Post by Werner Macho
Hi Alex,
Post by Alexander Bruy
I'm afraid it is not true, there are lot of issue even with LTR SAGA, see
for example https://issues.qgis.org/issues/17726. There are also some
recent threads in developers mailing list about broked SAGA and GRASS
modules.
ok is not perfect :) but is for sure is much better compared when we
tried to keep the pace with SAGA changes.
-- G --
--
Alexander Bruy
Paolo Cavallini
2018-02-01 12:21:57 UTC
Permalink
Post by Alexander Bruy
Hi Giovanni,
my point was that we still can not maintain hundreds of algorithms.
well, we did, more or less efficiently, until now.
I understand your point, however.
all the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
Loading...