With This, I Am Commencing My Submission For The #30DayMapChallenge For 2023 🗺

home is where you are

🌱 Google Earth Engine 101

Uploading a shapefile as an asset in GEE and making use of it

🟢 Beginner-friendly.

🆓 Free with no hidden monetary cost.

🤚🏻 Requires registration so sign-up 👉🏻https://signup.earthengine.google.com/, access via browser and Internet connection

🖥️ Available for Windows, Mac and Linux.

Google Earth Engine or lovingly called GEE is another free and open platform provided by Google to provide a very vast and comprehensive collection of earth observation data. Since Sentinel-2 is no longer available for download at USGS Earth Explorer, I find the alternative too challenging for me so GEE seems like the easiest way to go. If you're looking for a one-stop platform to access satellite imagery for free, GEE is a great place to start. You don't have to learn JavaScript explicitly to start using this tool.

Python: Geospatial Environment Setup (Part 2)

Hey again folks! I am here for the second part of Python environmental setup for a geospatial workspace. I published the first part of this post two weeks ago. So if you've not yet read that, I'll catch you up to speed with our checklist:

Install Python ☑

Install Miniconda ☑

Install the basic Python libraries ☑

Create a new environment for your workspace

Install geospatial Python libraries

🗃 Create a new environment for your workspace

Since we have actually manually set up our base environment quite thoroughly with all the basic libraries needed, to make our work easier, we can just clone the base environment and install all the additional essential libraries needed for geospatial analysis. This new environment will be called geopy. Feel free to use a name you identify most with.

Why don't we just create a new environment? Well, it means we have to start installing the Python libraries again from scratch. Although it is no trouble to do so, we want to avoid installing so many libraries all at once. As I mentioned in Part 1, there is always a risk where incomplete dependencies in one library will affect the installation of other libraries that you intend to install in one go. Since we already have a stable and usable base environment, we can proceed to use it as a sort of pre-made skeleton that we will build our geospatial workspace with.

1️⃣ At the Anaconda Command Prompt, type the following:

2️⃣ Press Enter and the environment will be clone for you. Once it is done, you can use the following command to check the availability of your environment 👇🏻

You should be able to see your geopy environment listed along with the base environment.

👩🏻‍💻 Install geospatial Python libraries

Here we will proceed with the installation of a few geospatial Python libraries that are essential to reading and exploring the vectors and rasters.

🔺 fiona: This library is the core that some of the more updated libraries depend on. It is a simple and straightforward library that reads and writes spatial data in the common Python IOs without relying on the infamous GDAL's OGR classes.

🔺 shapely: shapely library features the capability to manipulate and edit spatial vector data in the planar geometric plane. It is one of the core libraries that recent geospatial Python libraries rely on to enable the reading and editing of vector data.

🔺 pyproj: is the Python interface for the cartographic projections and coordinate system libraries. Another main library that enables the 'location' characteristics in your spatial data to be read.

🔺 rasterio: reads and writes raster formats and provides a Python API based on Numpy N-dimensional arrays and GeoJSON.

🔺 geopandas: extends the pandas library to allow spatial operations on the geometric spatial data i.e shapefiles.

💀 As you might have noticed, we won't be doing any direct gdal library installation. It's mainly due to the fact that its installation is a process that seems to be accompanied by misery at every turn and involved workarounds that are pretty inconsistent for different individuals. Does it mean that we won't be using it for our Pythonic geospatial analysis? Heck no. But we will be taking advantage of the automatic dependency installation that comes with all the libraries above. The rasterio library depends on gdal and by installing it, we integrate the gdal library indirectly into our geospatial environment. I found that this method is the most fool-proof. Let's proceed to the installation of these libraries.

1️⃣ At the Anaconda Command Prompt, should you start from the beginning, ensure that your geopy environment is activated. If not, proceed to use the following command to activate geopy.

Once activated, we can install the libraries mentioned one after another. Nevertheless, you also have the option of installing them in one go directly using a single command 👇🏻

💀 geopandas is not included in this line-up NOT because we do not need it. It's another temperamental library that I prefer to isolate and install individually. If gdal is a rabid dog...then geopandas is a feral cat. You never know how-when-why it doesn't like you and forces a single 10-minute installation drag to hours.

3️⃣ Once you're done with installing the first line-up above, proceed with our feral cat below 👇🏻

4️⃣ Use the conda list command again to check if all the libraries have been installed successfully.

🎉Et voilá! Tahniah! You did it!🎉

🎯 The Jupyter Notebook

It should be the end of the road for the helluva task of creating the geospatial environment. But you're going to ask how to start using it anyway. To access this libraries and start analyzing, we can easily use the simple and straight-forward Jupyter Notebook. There are so many IDE choices out there but for data analysis, Jupyter Notebook suffices for me so far and if you are not familiar with Markdown, this tool will ease you into it slowly.

Jupyter Notebook can be installed in your geopy environment as follows:

And proceed to use it by prompting it open via the command prompt

It ain't that bad, right? If you're still having problems with the steps, do check out the real-time video I created to demonstrate the installation. And feel free to share with us what sort of problems you have encountered and the workaround or solutions you implemented! It's almost never a straight line with this, trust me. As mentioned in the previous post, check out the quick demo below 👇🏻

YouTube

🌏 Python: Geospatial Environment Setup 🎉

A deconstructed time-lapsed Python geospatial environment setup with `conda` from downloading necessary software to loading the Jupyter Note

See you guys again for another session on geospatial Python soon!

train on water

Floating...in the mid of it

Split by Attributes GP tool....when would you actually use this?

There are times when you're making a map but symbolizing using the symbology feature is not enough to characterize the data visually. Thus, having this tool makes cartographical work a little easier by generating copies of the original data, split into separate layers based on the attribute that we need. By doing this, it makes the task of adding the legend much easier in the layout as well.

Most often, when making maps for slide presentation, you would want to segregate data into separate layers with certain uniform values for a certain attribute and a create a new data layer which we can use over and over again.

Although definition query can help with visualizing and showing the features with the attribute value that we want, we may want to create a separate data to avoid compromising the original data or constantly repeating the task of typing/configuring the SQL commands.

This tool is valid for shapefiles and feature classes. Any other data types may need to be converted into those two formats before you can run it. Check out the long-winded demo below:

Since this tool is actually a Python script, it can be integrated into a code for batch geoprocessing or model for iteration over many data layers or interconnection to other tools; automation at its full-on glory! 😁

how to read research paper effectively 📚

I'm hitting the backed-up reading list that I've accumulated in my Zotero. It's annoying and you procrastinate the task of reading as much as possible when you're in that potato phase. I am demotivated, bored, constantly tired, and feel like devoting myself to reading storybooks for life. If I can get paid for all the hours I sleep every time I feel like signing out from life, I could be making a decent living. But, too bad, I don't.

I do not endorse any products or review anything since I feel like, to each, your own. So, I'm not going to tell you what works best or how some tips can magically fix your life. I am lucky that I have an incredible academic supervisor, a flexible boss at work, a very academic-oriented sibling, and a supportive squad of friends. Even with all that, I am still depressed. So, if you're down on the low at the moment, you're not alone. But when you have made a promise, you will look like a total flake if you don't deliver. So, you gotta move your ass anyway, right?

I just started reading papers again and it was so hard. Two weeks go by without me making any progress...just stuck on one paper and not retaining a single piece of information at all. All that forehead and nothing...nothing sticks. So you can say that I am hating life right now. But, today...I manage to reach some sort of compromise with myself and it starts to feel good. So, I would like to share it with you guys who could be struggling to get the engine started as well.

🎯 Literature Review Catalog

My supervisor is an awesome human being. He's the manager/cheerleader/mentor/Allfather/Captain America/Britney Spears to my lackluster academic history. He had been keeping tabs on me despite my intermittent anxious mood that swings like a freaking metronome, so you can say that he practically keeps my boat afloat at this unprecedented time. For our proposal writing (there's a whole army of us that he's supervising), he shared something valuable. The 'Literature Review Catalog'.

Yes. It's an Excel Sheet. Nothing fancy with very normal columns that indicates the papers/resource you've read. Looks simple and useful. The columns are populated as follows:

Year: The year of publication.

Author: Short author list.

Country (Study Area): The areas that are being studied in this research. If you're an Earth Science student like me, you can narrow it down to countries. But I think overall, countries are the most general part of discriminating different studies.

Main Keyword: I create my own keywords to develop my own system of comprehension. But I do create a column for the keywords found in the paper itself.

Issue & Objectives: You can find this information from the Abstract and Introduction part of the paper.

Proposed Method: This can be found in the Results section but I usually scan through the Methodology to add in more information when I do second round scanning of the paper.

Findings & Conclusions: I add in more notes on information that is new to me here in addition to the conclusion. New information can be extracted when you do another once-over of the paper and a conclusion can be obtained from the Conclusion section.

Reference: You can find references that are relevant to your studies from this paper! So why not? Right?

But, it's the laborious work that comes with it that turns my stomach. It scares the hell out of me despite any motivational speech I give myself. But it can all make sense when you pair it with the following method 👇🏻👇🏻👇🏻

🎯 How To Read A Paper Quickly & Effectively | Easy Research Reading Technique

This is the gem my sister told me about yesterday. I brushed it off since it stresses me out to see people sharing their speed-reading techniques, study tips, and how to ace all the subjects in the world or how to get a 4.0 GPA. It really isn't the good people's fault and I blame it on my constantly anxious self. I don't even know what's wrong with me, so...it's not them. It's me. But, here, we're gonna work on 'me'. So, give this 10 minutes video a watch. It's worth it because Dr. Amina Yonis really knows what she's talking about and what's even better, she really is an advocate for effective reading/studying. It's short enough for you to maintain your attention span and you will learn how to actually 'evaluate' your reading materials; are they worth the second shot at reading? Is there any added value to it?

To summarize, what you should look out for:

Title: Read the title and find the keywords

Abstract: Lookout for the results and methods in a simple sentence

Introduction: Read the first and last paragraphs. Most of the time, the first paragraph highlights the satellite view of the crisis and the last paragraph zooms straight for the objective.

Results: Pay attention to the headings since that more or less highlights what was it that they find. If there aren't any headings, try looking at them by paragraph. Scan them through.

Conclusion: This summarizes everything in the research paper.

After the 'Conclusion', you may feel like it is an info/findings that you've already expected or grasped, and you may just proceed and read other new ones in your pile. But if you need to dive deeper, jump to the 'Results' again for the key figures or results and limitations.

So ...

How do you go about reading this and what has it got to do with the 'Literature Review Catalog'? Well, using this efficient reading method and taking out the notes into the columns will help you condense all the important information and helps you stop re-reading constantly the details that are not paramount to your study.

🎯 Forest App

To amp up and see if it was effective, I actually timed myself with the 'Forest App'. I have been estranged from it since my potato phase, but now, it's back to being that BFF I need. It took 10 minutes to go through all the steps and if the paper isn't heavy-laden, 5 minutes to fill it into the 'Literature Review Catalog'. I manage to think and ask questions in my head as I fill in the columns and I believe that's the most important part of the effective reading that we need as someone who's jumping into a very dynamic environment of scrutinizing existing work. You can use any sort of timer to actually give a sense of urgency to your work - it does help to a certain extent. So, if you intend to have fun creating a forest of pretty trees while making good of your focus time, check out this video!

🎯 Reference Manager

And please please please, organize/record your references responsibly using reference management software. Some swears by Mendeley, or the good ol' EndNote. There's also Flowcite and Citationsy. Use them. Don't download those papers indiscriminately without recording the details that can help you sync them straight to your word processor using viable plugins. I personally use Zotero. It comes with a Chrome plugin and Microsoft Word plugin that you can download separately. It's compatible with Linux and iOS operating system. I used to park my work at Mendeley, but I find Zotero more powerful and flexible enough to use and it actually helps me to make the effort to remember what I actually downloaded rather than rely on the convenience of going back and forth to cloud storage. And it's open-source. So, try it out to create an organized library.

To all the aspiring scholars out there, when you win, we all win. Share your phase and troubles with the #studyblr or here with me. Emotional support is important and if the internet does not give you peace of mind, sign out and unplug. It's ok. When you're ready to work, reach out to anyone you think will respond positively and want to help you succeed. We can't all do things alone. So, start that power-up playlist and start working!

survey123 offline

Survey123 for ArcGIS is perhaps, one of those applications that superficial nerds like me would like; it's easy to configure, kiddie-level degree of customization with 'coding' (for that fragile ego-stroke) and user-friendly template to use. 

No app development/coding experience is required to publish a survey form and believe it or not, you can, personalize your survey to not look so meh. 

It took me some time to stumble through the procedures of enabling this feature before I understand the 'ArcGIS Online' ecosystem to which this app is chained to. 

So how do we do it? And why doesn't it work pronto?

This issue may be due to the fact that when we first start creating our forms, we go through the generic step-by-step procedures that leave little to imagination what was happening. Most of the time, we're too eager to find out how it really work. 

When we publish a Survey123 form; be it from the Survey123 website portal or the Survey123 Connect for ArcGIS software, we are actually creating and publishing a folder that contains a hosted feature layer and a form. It is on that hosted feature layer that we add, delete, update or edit data it. From ArcGIS Online, it looks like any feature service that we publish out of ArcGIS Desktop or ArcGIS Pro, save for the special folder it is placed in with a 'Form' file. 

To enable any offline function in any hosted feature layer in ArcGIS Online, you will need to enable the 'Sync' feature. So far, in many technical articles that I have gone through to learn how to enable this offline feature always goes back to 'Prepare basemaps for offline use'. It is a tad bit frustrating. But my experience when deal with 'Collector for ArcGIS' gave me the sense of epiphany when it comes to Survey123. So when you have prepared your Survey123 form for offline usage and it still doesn't work...do not be alarmed and let's see how to rectify the issue. 

1. Locate your survey's hosted feature layer

At your ArcGIS Online home page, click 'Content' at the main tab. We're going to go directly to your hosted feature layer that was generated for your survey when you published. 

Locate your survey folder. Click it open 

In the survey folder, navigate to the survey's hosted feature layer and click 'Options' button; the triple ellipses icon

At at the dropdown, click 'View item details'. Please refer to the screenshot below: 

2. Change the hosted feature layer settings

At the item details page, navigate to the 'Settings' button at the main header and click it. This will prompt open the settings page for the feature layer. Refer to the screenshot below:

At the 'Settings' page, there are two tabs at the subheader; 'General' and 'Feature layer (hosted)'. Click 'Feature layer (hosted)' to configure its settings.

At the 'Feature layer (hosted)' option, locate the 'Editing' section. Here, check the 'Enable sync' option. This is the option that will enable offline data editing. Please refer to the following screenshot: 

Don't forget to click 'Save'

With this, your hosted feature layer which serves as the data model is enabled for synchronization. Synchronization helps to sync back any changes you've made when you're out on the field collecting data; editing, adding, deleting or update...depending on what feature editing you've configured. 

It's pretty easy once you get the hang of it and just bear in mind that the data hierarchy in the ArcGIS Online universe are as follows:

Feature layer (hosted) > Web map > Web application

Once you get that out of the way, go crazy with your data collection without any worries!