Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Old issue OCNA migration to new code #1973

Closed
TermiteHunter opened this issue Feb 16, 2019 · 60 comments
Closed

Old issue OCNA migration to new code #1973

TermiteHunter opened this issue Feb 16, 2019 · 60 comments
Assignees
@harrieklomp @andrixnet
Labels
General Discussion OCNA migration temporary label to fast find issues Priority Critical
Milestone
OCNA migration

Comments

@TermiteHunter
Copy link

Hello and forgive me if speaking out of turn.

It is now Feb 2019 (2 years since the last comment here on the migration) and we are still operating on an even more broken site.
There are failures on our site that affect all users and discourages new members.

The attempt to bring OCNA into the new PL code failed long ago when we were abandoned and forgotten. Attempts to get help with a PL or DE styled update by Mr-Yuck have been met with a few promising ideas from various individuals only to be abandoned before they begin.

The OC Community is about to lose OpenCaching North America. (Mexico, US, Canada and parts of Central America)
The site looks and operates poorly, an embarasment to the OC community as a whole.
This can not continue. I find that I can no longer encourage others to explore OCNA in it's cuurent state and have even declined to list my own caches on OCNA (and I am an Admin there)

Please contact Mr-Yuck and re-instate the OCNA migration to an updated version of the OC code so that we may continue as a valued and valid OC site.
Need is URGENT.
Admin at OCNA are preparing to allow this node to join Navicache and become a non-entity if help is not recieved soon.

Action is needed.
The site is failing. (maps fail. search fails, registration fails)
Admin is discouraged. (too many promises with no follow-up and abandonment)
Use is plummeting. (new users down, listings down, logs down)
Failure is imminent. (by code failure, lack of use, or intentional failure to renew domains or hosting for a failing site)

If the goal is to have nodes in all parts of the world, The community is failing a huge part of that world.
Save OCNA NOW
@kojoty
Copy link
Member

kojoty commented Feb 16, 2019

hi @TermiteHunter

Let's open a new discussion and try to conclude where we are now.

First:
OCPL as a whole has also its problems. There are only a few developers who works on maintenece and development of OCPL code. We are in a long run to the refactoring of the "old" code and there are still much work to do.

Second:
What is your (OCNA) current situation - how many technical people are there now - what kind of support you need and expect and what you can ensure yourself. I mean if there is someone who has time and skills to cooperate with us in this process of migration OCNA to OCPL code.

@Mr-Yuck are you there? What's your opinion?

@deg-pl
you have experience with resurrection of OCUK.
Do you think we are able to do the same for OCNA?

@deg-pl @harrieklomp @following5 @andrixnet @mzylowski
What do you think? Do you think we can cooperate now to help with OCNA migration to OCPL code?

If you decide to help I'm in.

@kojoty kojoty mentioned this issue Feb 16, 2019
Closed
@deg-pl
Copy link
Member

deg-pl commented Feb 16, 2019

I'm pro. I can help.
But we have one big issue - we don't support states and US post codes. Nor in caches and search engine.

Rest should not to be a big problem. I can make new OC NA server and probably to migrate existing data. I'm only busy for a few next days.

@following5
Copy link
Contributor

Subissues:

Special tables in OCUS database:

  • ca_province
  • cache___list
  • geometry_columns
  • mx_states
  • spatial_ref_sys
  • tl_2009_us_zcta5
  • us_layer
  • us_parks
  • us_state_c

@harrieklomp
Copy link
Member

harrieklomp commented Feb 16, 2019 •
edited
Loading

@deg-pl @harrieklomp @following5 @andrixnet @mzylowski
What do you think? Do you think we can cooperate now to help with OCNA migration to OCPL code?

Yes, let us help OCNA with this great OCPL code?
This shows that the staff behind this great code also are great :-)

@TermiteHunter
Copy link
Author

Thanks for the response. I am trying to get MrYuck involved. years of no results has left him a non-frequent visitor to OCNA and Geocache related e-mail accounts. He will pop up sometime.

To First question: We are aware of ongoing issues with PL. That was stated as a partial reason the upgrade was terminated 2 years ago. Let's face it, There will always be some issues to contend with but a basicly functional site with at least the appearance of being maintained is prefferable to OCNA as it is.

To second question: We have NO tech people. We can find the time but have no expertise which is partially why we are this situation. We just want a site that members can actually use that looks to be maintained. Graphics for some items not used in Europe were already provided.

I'm pro. I can help.
But we have one big issue - we don't support states and US post codes. Nor in caches and search engine.
We currently have this ability in the old code. I believe that much of this may have already been done or at least started previously. It doesn't have to be by zip code (it would be nice and preffered) Something similar would be better than only a single page being returned with following pages failing to resolve. This occurs on all searchs, zip or not.

Thanks again.

@kojoty
Copy link
Member

kojoty commented Feb 16, 2019

@TermiteHunter

we need to know if we can start:

  • do you have access to any hosting account?
  • who controls the OCNA domain?
  • do you have an access to current server - we will need to copy pics etc.

then:

  • @deg-pl is able to prepare new server configuration for you
  • in the meantime we try to identify and if possible fix the incompatibility in code and configuration

@TermiteHunter
Copy link
Author

TermiteHunter commented Feb 16, 2019 via email

@Mr-Yuck
Copy link

Mr-Yuck commented Feb 17, 2019

C'mon TH, it's not THAT difficult to get me to appear. Andrei was helping us, and more specifically, we were talking about a quick "OCUK type" fix. But I haven't heard from him since mid October. I believe he does have a test server running with OCUS mostly migrated. We have a web hosting company out of Montreal Canada, and of course I have access to the server, which I interface with with cPanel. I haven't used ftp in probably 3 years, although you can set it up with cPanel. As noted at the very bottom of the page on opencaching.us, it is currently running with Litespeed web server (not apache), CentOS (not Ubuntu), Mariadb (not MySQL) and Exim internet mailer. Let me know what you would need. And I think Andrei should definitely be consulted first, to see if anything he's done may be of use.

@kojoty
Copy link
Member

kojoty commented Feb 17, 2019

@andrixnet are you there? (Andrei is here but it seems he has been busy for last few months)

I think that the best solution would be the same as for OCUK: prepare the new server - I don't know how many traffic you have right now but I expect that cheap VPS would not cost much more. In my opinion this is much better because staying with completely different software stack can caused additional issues, which can be difficult to debug for us - I believe @deg-pl agrees with this - am I right?

In the meantime today I will try to review issues which andrixnet marked as a important for OCNA today.

@deg-pl
Copy link
Member

deg-pl commented Feb 17, 2019

Yes. I agree with @kojoty
Best option is new VPS in KVM virtualisation (not OpenVM!!!). In google I've found many cheap VPS in USA (under $10/month). OC UK use cheap SmallVPS https://www.arubacloud.com/vps/virtual-private-server-range.aspx

Let's try to find which parts of our code doesn't fit OC NA and set priorities. I think for example, that "states" support is important (maybe our regions will fit?) but apporiving all caches by OC Team is not such important for now.

@Mr-Yuck
Copy link

Mr-Yuck commented Feb 17, 2019

Andrei (who we definitely must consult with) had actually mentioned last October that he had enough server space to host us for free? We were originally set up on the U.S. web hosting Company Rackspace, which was costing $40 U.S. a month! We pay $16 U.S. a month now, and this company (Kickassd) given server access to Rackspace, moved the site over for free. I concur with all migration issues mentioned here, and I will say every one of them have been discussed with Andrei at one time or another.

@andrixnet
Copy link
Contributor

In part it is my fault. I've been in contact with one of OCNA admins (Mr. Yuck I think, I also knew him as "Jim"). He has given me access to OCNA hosting several years ago and I helped him out with some small bugs/problems over time.
OCNA was based on a modified version of the old OCUK fork. I suggested migrating to current opencaching-pl code, and to that end I initiated project OCNA migration here. Also, highly related, is project Well defined data types because of the differences in geocaching activity and many remnants of old DB based data types in opencaching-pl code.

While trying to make heads and tails of the migration necessities, my progress was slow.
In 2018 I set up a devel server of my own for both OCRO and OCNA, on which to test my stuff.
Since then I lost contact with Mr. Yuck.
Also, since the birth of our second child, my time for this project shrunk to zero (OCRO also severely affected; @following5 I still have to implement some of your instructions).

@andrixnet
Copy link
Contributor

That's it with the history. Now to the practical.

As I said, I have a devel server up and running for OCRO and ready for OCUS, all but the database conversion.
I can offer it as hosting for OCUS working on opencaching-pl code at no cost.

The existing hosting of OCUS is big mess, at least configuration wise. I still have access to it.
I have a complete backup of all OCUS resources (files and databases):

  • OCUS opencaching site
  • forum
  • wiki
  • blog
  • bitcaching page
    These were made december 2018.
    Based on these I should be able to recreate them on a separate hosting, should Kickassd hosting cease to function.

I think I can make another set, as long as the Kickassd hosting is still up and I have access.

@andrixnet
Copy link
Contributor

Technical-wise, project OCNA migration describes most requirements. I'll try to summarize them again here, to the best of my knowledge:

  1. OCNA specific cache types

  2. OCNA specific attributes
    This was hardest for me, since cache types, sizes, attributes and a few other resources are still scattered in both code and database for opencaching-pl

  3. region assignment based on US states (and Canada and Mexico).
    I have the OCUS database, which contains geographical data similar to our nuts_layer and nuts_data resources. This should not be too difficult to convert. I made partial progress, keep in mind that I also wanted to convert al geographical data to polygons (instead of linestrings) so that ST_WITHIN MySQL/MariaDB function returns the proper result. (see here region from coordinates yields nothing #120)

  4. detect/store/search by ZIP code.
    OCUS database contains similar geographical data regarding ZIP codes, this would be a totally new implementation in opencaching-pl. Either use that data, or add a dependency on an online service, other then Google, of course, due to costs.
    This last one, while I understood may be important to US users, used to using ZIP codes, is hardest and most time consuming to implement.

@andrixnet
Copy link
Contributor

Pssst, the kids are sleeping, so I'm busy writing here.
Harrie kept me informed from time to time about OC developments.

I'm looking up the stuff I already did for OCUS on my computer and reviewing it.

@following5
Copy link
Contributor

following5 commented Feb 17, 2019 •
edited
Loading

@andrixnet

  1. OCNA specific attributes
    This was hardest for me, since cache types, sizes, attributes and a few other resources are still scattered in both code and database for opencaching-pl

Cache sizes meanwhile are completely migrated to code; you just need to add Config/geocache.us.php to configure the OCUS sizes.

For attributes I can do the same, as soon as the DB update system is working at OC RO (see #1943 (comment)).

Then only the cache types issue is left => #860.

@Mr-Yuck
Copy link

Mr-Yuck commented Feb 17, 2019

@andrixnet Mr.Yuck is definitely Jim. :-)

@TermiteHunter
Copy link
Author

TermiteHunter commented Feb 17, 2019 via email

@kojoty kojoty added the OCNA migration temporary label to fast find issues label Feb 17, 2019
@andrixnet
Copy link
Contributor

andrixnet commented Feb 18, 2019 •
edited
Loading

cache_status is another table containing data type definitions
waypoint_type is also.
log_types is also

@following5
Copy link
Contributor

following5 commented Feb 18, 2019 •
edited
Loading

cache_status is another table containing data type definitions

Also log_types and waypoint_type. But I don't think that any of these have special contents at OCUS, which need special care when migrating to current OCPL code. (log_types definitely has not ... whoop some minor differences found, but not relevant for migration). Attributes also are uncritical - GPX already is prepared for OCUS attributes, and the rest of code uses the cache_attrib table contents.

I think the cache types are the only of those entities that need special migration.

@andrixnet
Copy link
Contributor

OKAPI attributes.xml might need some tending (?)

@andrixnet
Copy link
Contributor

@kojoty Yes, there is. SQL tables with LINESTRING shapes. A bit simpler then NUTS data, less hierarchy, in separate tables for US, MX, CA.
It would be of great help if you could sort them out, also sort out some other migration data. At some point I am not sure how to proceed.

I can show you my work. I'll contact you privately with details.

@TermiteHunter there seem to be more email subsystem problems that I will have to sort out.
Do you have fellow cachers with a yahoo account to ask to try it out? (it worked for me, so far, with yahoo).

@Mr-Yuck
Copy link

Mr-Yuck commented Mar 29, 2019

I will try with a yahoo account.

@andrixnet
Copy link
Contributor

In contact also with "Bon Echo" OCNA admin.
Fiddled a bit more with email subsystem settings to improve deliverability.

@harrieklomp
Copy link
Member

Just updated the software with "git pull", so we can see the actual state on http://ocus.dev.andrix.eu/index.php .

@harrieklomp
Copy link
Member

A small update for OCUS devel site.

  • Maps are working. Need little fine tuning to show correct area. (coordinates and zoom)
  • Attributes are shown only with English text. (DB)
  • Added a copy of "contact" page". Is now ready to translate. Now only visible in English language.
  • Most links are linking to correct pages, like forum and wiki. Some wiki links do not work because there is no wiki pages available.
  • Waypoint types are complete now. (DB)

These where the easy things to do. The harder things i leave to the more experienced persons :-)

@kojoty
Copy link
Member

kojoty commented Apr 7, 2019 •
edited
Loading

@harrieklomp and the rest of collegues :)

These where the easy things to do. The harder things i leave to the more experienced persons :-)

Please create one updated list of issues which we still have in this subject

@andrixnet
Copy link
Contributor

  1. Most important of all: cache types.
    OCUS has a larger list of cache types then most other nodes and an additional problem is that some cache type IDs overlap.
    This I researched thoroughly before and the complete documentation can be found here: https://wiki.opencaching.eu/index.php?title=Cache_types
    There are several implications:
  • there are overlapping cache type ID numbers between OCPL, OCDE and OCUS.
  • IMO all cache type IDs must be unique, therefore I propose changing IDs for types that have overlapping IDs and assign new unique IDs to new types from OCUS
  • code should be updated to reflect these changes.
  • when code is updated, a database update must be deployed as well, to change the resolved duplicate IDs on all opencaching-pl based nodes.

  1. Support for mascot image in header, besides the logo.
    oc1

  2. Additional top menu items and their respective pages (from http://www.opencaching.us/index.php)
    I think Harrie can handle it.
    oc2

  3. Home page welcome text
    I think Harrie cand handle it.

  4. Additional items in MAIN MENU and their respective pages
    oc3

  5. Additional links on the left menu:

  • OC network links
  • donate link
  • mascot
  1. very important game parameter:
  • OCUS counts event "attended" to total finds.
    IMO this should become a per/node configurable parameter, by default OFF, but set to ON for OCUS.

  1. USA, Canada and Mexico geo data import from old database in a compatible way to our NUTS data.
    Part of it I am trying to handle myself, work in progress. Where I get stuck, I'll ask for help.

  2. Search by ZIPcode.
    ZIPcode geo data import from old database is easy, but support for search by it (PHP code) I cannot handle.

  3. USA, Canada, Mexico regions definitions. Probably part of 8)
    Also correlation with existing cache data

  4. setup of Powertrail module.
    I think Harrie can handle it.

  5. set website icon (see Missing favicons #1547 )

  6. setup of cron jobs
    I will handle it.

  7. setup of automatic updates
    To be done after migration and move to production.

  8. french and spanish translations

  9. how to handle differences between british english and american english.
    If these differences account only for a few cache attributes, then perhaps the attribute can be named something like this:
    Quick cache / One-minute cache

If there are more such differences, not only attribute name (which can be handled easily), then another solution must be designed. The only one I can think of is to maintain 2 sets english strings (en-us and en-gb).

  1. challenge caches is a OCUS specific thing that they did upon on unknown cache type. Perhaps this deserves it's own type.

  2. the page "phone apps for OC" (from main menu) I am sure might be of interest to all OC nodes

  3. Perhaps a simple online editor for articles, similar to news, available for node admins (of use for all OC nodes, see Pages editor for OCPL #1946 )

These are most of what I can think of at this time.

@andrixnet
Copy link
Contributor

Oh, and another very important issue:

  1. convert passwords from old cca 2009 opencaching-pl code to -current authentication scheme and the new salt and hash scheme.

This is very important to allow seamless transition for the users, instead of requiring each user to do a manual password reset.

@harrieklomp
Copy link
Member 

1. convert passwords from old cca 2009 opencaching-pl code to -current authentication scheme and the new salt and hash scheme.

This is very important to allow seamless transition for the users, instead of requiring each user to do a manual password reset.

This converting does go automatic.
If logged-in or generate new password there will automatic will add a "salt" and "hash".

@TermiteHunter
Copy link
Author

Thanks to Harrieklomp I have managed to login to the Dev site.
My email was changed and somehow that allowed me to login using my regular credentials without a password change.

I have yet to explore the site fully.
This may have been mentioned before but I did not see it,
The main page MAP OF CACHES still goes to the non-functional google map.

Thanks to all those participating in the OCNA update. You are doing great work.

@harrieklomp
Copy link
Member

The main page MAP OF CACHES still goes to the non-functional google map.

If i remember correct there is only one page with old maps. See http://ocus.dev.andrix.eu/myroutes_add_map2.php.
All other maps are the newest version.

Perhaps used by accident a link to another site?
BTW, in the Menu it is called "Cache map"

@TermiteHunter
Copy link
Author

TermiteHunter commented Apr 28, 2019 •
edited
Loading

Must have been some sort of other issue that gave me the old Google map. I now wee the correct current map when clicking Cache Map.

Low priority: QR code needs a OCNA version. currently shows OC.PL as url and has the OC sun logo.
Request that the OCNA North America version logo and Opencaching.us url be used.

Future suggestion for all OC very low priority.: Foreign waypoints boxes should remove no longer active Navicache.

_Find geocaches on: gpsgames.org | oc.cz | oc.de On cache page should reflect local options to OCNA. (gpsgames.org, geocaching.com, terracaching.com)

American / British English: unified terms would simplify issue. Select best option. Where there is apparently no cross British / American term that is suitable allow for alternative appropiate term.

@TermiteHunter
Copy link
Author

Just an update on the current "old" site to encourage completion of the upgrade.

We are essentially completely shutdown.

The hosting site did some sort of server migration that completely broke OCNA.
At first glance it looks fine but none of the key pages beyond the main page will load.
We are getting a hundred+ OKAPI errors daily.
I am unaware of what is being done if anything to solve the issue by MrYuck or the Hosting site.

@andrixnet andrixnet added this to the OCNA migration milestone Jun 20, 2019
@andrixnet
Copy link
Contributor

I suspect the hosting site changed PHP version (most likely explanation). Old code can no longer run properly, so migration has suddenly turned to the highest urgency level.

I'd like to propose July 1st as due date for switchover.
Many details to be worked out, see my email I sent directly.

@kojoty
Copy link
Member

kojoty commented Jun 20, 2019

Hi,

I talked with Deg.
We both think that it would be good to just prepare the new server ( same as for ocuk before) with fresh ocpl code, make ocus run as is and then try to port all their old features.

Please be sure that you save the last working day dump + files from the server ( pictures etc.)

Maybe just add some anonucemnt and make the current side RO or at least note that any changes from that moment will be lost.

Me and Deg are going to involve in the process after the weekend - now we are still on some holidays trips...

@andrixnet
Copy link
Contributor

At this time I am preparing the DEVEL/test instance to move to production.
Dataset is about 6 months old ad I do not have access to the latest data (some changes at their old hosting)
If I can retrieve some data later, I will try to integrate it afterwards.

For now, getting OC and utility sites working is highest priority this weekend.
Proposed deadline to production is July 1st, at the latest.

@bekuno bekuno mentioned this issue Jul 3, 2019
Closed
@TermiteHunter
Copy link
Author

Thanks for the upgrade. At least most of the site is functional now.
There are several parts that do not function such as sections of the left menu.

SOmething that has come up repeatedly is users requesting deletion of their account.
There appears no way for users to self delete and they are directed on the profile page to email admins to do it.
We appear to have no way to do this either.
Am I missing something?
How is this to be accomplished?
Users asking to be deleted is not an uncommon issue. I get emails with the request frequently.

@kojoty
Copy link
Member

kojoty commented Feb 20, 2021

Hi,

For now OC don't have any mechanism to remove user account automatically.

At least OCPL works like that:
user asks for removing his account. Then if user has any logs or caches account is locked, name of the user is anonymized and email address is removed from DB. If user account is without any logs or caches it cen be just deleted. All operations we do manually in DB (there is no many requests so it still can work like that).

Yes, we should think about script to allow user to remove/lock his/her account - I even think that maybe I will be able to work on this this later year.

Regards.

@TermiteHunter
Copy link
Author

TermiteHunter commented Mar 3, 2021 via email

@Amberel
Copy link

Amberel commented Mar 3, 2021 via email

@kojoty
Copy link
Member

kojoty commented Mar 3, 2021

OK, I try to work on this on next days - it shouldn't be too complicated...

@kojoty
Copy link
Member

kojoty commented Mar 3, 2021

I added the new issue on user account deletion here: #2226
I assume this issue can be closed now - please open new issue for each detailed topic to have some order here :)

@kojoty kojoty closed this as completed Mar 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@harrieklomp harrieklomp

@andrixnet andrixnet

Labels
General Discussion OCNA migration temporary label to fast find issues Priority Critical
Projects
None yet
Milestone
OCNA migration
Development

No branches or pull requests
8 participants
@harrieklomp @following5 @kojoty @andrixnet @Mr-Yuck @deg-pl @Amberel @TermiteHunter