An example: Calculating a field value on a FeatureLayer

- [Instructor] Feature layers, like these in this ArcGIS online map, can be edited and analyzed with the ArcGIS API in a notebook. I'll show you how to use the Python API to add a field to an attribute table and then calculate a value into that field. We've got a feature service here that shows homes in a small town. The homes have been categorized based on whether they're currently occupied, vacant, or dilapidated. We've got an inspector who's been going out in the field to inspect the homes, and we just want to write a script to put this inspector's name into the table for the homes so we know who's inspected them. So let's see how it works. So I've got a project open here in ArcGIS Pro and I've already got a notebook started. The first cell connects me to ArcGIS Online and the second cell goes out and grabs that feature service from ArcGIS online using its URL. Of course, I can easily create a map and visualize that layer. I'll create the map with g.map, then I'll add another cell down here and just add that layer in. That's m.content.add, right? So we can see the layer up here on the map. I just feel it a little more comfortable looking at the layer as I begin to process it. Of course, we can easily work with the properties of a feature layer, like to find out what the geometry type is. So I'll add another cell. Whoops, we get that cell down here at the bottom. I'll add another cell. And in this cell, I'll just print out the geometry type, which we already know it's points, we can see it right? SturgisHomes.properties, square bracket for a dictionary key, and my key is geometryType with a capital T. Have to check the help on that to be sure. So it says it's a point feature class, which we knew. If I have right permission to this service, I can use the feature layer .calculate method to update values in its attribute table. So that's what I want to do. But first, let me find out what the column names are so I can see what I have to work with. So I'll add another cell. So first, let me list out the column names so I can see what I have to work with. So I'm going to use sturgisHomes.properties, the same place where I got that geometry type, right? But there's a different key in that dictionary called fields, and that'll give me a list of all the fields. It actually gives me a list of the fields and some field properties like data type. So it's actually a dictionary itself, right? The dictionary is a whole list, right? It's a bunch of fields, or at least several. So I'm going to put this whole thing in a loop. For each field in sturgisHomes.properties['fields']. And then I'll just print each field the name key. So I'm asking for the name, the value of the name key in the Fields dictionary, which is a sub dictionary of the properties dictionary. And so those are my column names. Okay, so let's say I want to add a new field to the attribute table to hold the name of this inspector. I'll need to define the new field. You know, its name, data type length, just like if you were adding a column inside Pro. I do that with a dictionary. So let me add a new cell. I'm going to create a variable here called new_field just to hold the dictionary definition of this new field. I'll open the curly braces to say I'm about to give you a dictionary. And then I have to put in the key-value pairs. So the first key that I need is a name, and I'll put Inspector as the column name. And then I put a comma, 'cause these are comma-separated values in this dictionary. The second one is type, and the field type is going to be esriFieldTypeString. It's going to take a second to make sure I have that spelled right. EsriFieldTypeString. Yep. And then, again a comma, and then the length is going to be 50, and then I close off the dictionary. Now, how do I know this? Again, I got to check the help to see what the structure is for defining a new field. You can't just magically know this, right? So don't feel bad if this looks like a different language to you right now. That you got to check the help and see the syntax. I'm going to go ahead and run this. Now, I have this new variable called new_field, right? I can just print it out. It's just text. Right? It's a little dictionary. Now, the field is added with the content manager using the add_to_definition method on this layer. So it works like this. Let me add a new cell. I'm going to make a variable. I'll just call it add_field_result. This is going to return a true or false, saying whether or not my add field was successful. So I'm going to use the method. So sturgisHomes, that's my feature layer, .manager, that's my manager class, .add_to_definition. I have to pass a dictionary into this method because it actually allows me to add a bunch of fields all at once, but I just want to add the one. Remember, I stored its definition in that new variable called new_field. So inside the parentheses, I'm going to put up curly braces for a dictionary, and then I'll say, I want to add fields, that's what I want to add to the definition, and then new_field, that's my variable. So let's talk about the syntax a little bit. Add_to_definition lets me add lots of things to a feature layer. So I could add fields, I could add domains, I can add other things. So I have to put a dictionary in here with a list of all the things I want to add to this particular feature layer. I only want to add fields, so that's my key, and then my value is the definition of that new field that I specified up here higher. Now, there's a little catch here. If I just try to print out the columns from the attribute table again, I won't see this new field yet. It worked, but the version of the feature layer that's here in notebook in memory is still the original version, not the version that I just modified. We can see it. I'll just copy this guy up here, right? I could just go up there and rerun it, but I'm going to bring it down here. I'll rerun this guy just to be sure, and see there's no inspector added in there yet. But because that's still the original version, it's not the one that I just modified. So I have to go out and grab that feature layer again from ArcGIS online. So the column got added to the dataset in ArcGIS online, not to the one in memory right here. And I know that's really weird, but that's how it works. So I have to go grab it again. So I can just go get that code up here from when I first created the feature layer, exact same code. And I could just go rerun the cell again. That would be okay too. But I just don't want to confuse you guys in the video, right? So I'll just run this new cell, but it's exactly the same code. And then I'll just go up and rerun this, print the fields. And so as you see now, there's an Inspector field, 'cause I went out and got the data set that I had modified. I know that's really weird, right? Seems like if you would do an add field, it would be happening here in memory. Just not how it works. Okay, so now that I have the new field, we can calculate the new value into the field. I'm going to add a new cell. And in this new cell, I'll do the calculation. So I'm going to do sturgisHomes.calculate, and then I'm going to set that equal to a variable also. The variable, calc_result, is not going to hold any new data. It's just going to hold a true or a false, telling me whether the calculation worked. The values will actually be written directly to the feature layer. I'm going to put a query in here that looks kind of strange. So the first parameter for the calculate is a query, saying, which records do you want to modify? In this case, I'm just playing, so I'm going to modify all of them. If you want to modify all of them, you put this where="1=1" in there. It looks strange, but it's a standard query that says get all the records. Don't think too hard about it. Just remember, if I want all the records, that's what I need to put in the where clause. Now, I'll add the calculate expression as the second parameter, calc_expression =. Oops, got to spell it right. Calc_expression =. Now, what you put into this calc expression is, guess what? Another dictionary. So I'm going to put a set of curly braces. And the dictionary has the column name and the value. And it's got two keys. There's the field key and there's the value key, okay? So I'm putting in a field and a value. So my field is going to be the Inspector, and the value I'm passing in is going to be, let's just say, how about Alphonso? And then I close the dictionary and close the parentheses for the parameters. So that calc_expression is kind of strange-looking again, but you're passing in dictionaries, a key, a value pair, one for the field name, one for the value. Okay, so I can run this. Oops, I missed the close parentheses. Sorry about that. Get rid of that extra parentheses and run it again. Now, it's not going to tell me that anything happened. It's just doing something in the background. If I add a cell and I print calc_result, I'll hope that it'll say true, meaning that the calculation worked. Success true. So I could see it by bringing it into a map, or querying its records, or whatever I might want to do. So, for example, if I go back here in map and I go over to my portal and I look for my data, and I find that Sturgis_Kentucky layer and drop it on my map, I should see that new Inspector column, right? So I'll go down to the Holmes attribute table. It should have an Inspector column and it should be populated with Alphonso. And there we go. Oops, not there. There, there's the Inspector, and Alfonso has been busy with all 660 records. But if I'd put something in that where clause instead of a 1:1, I could have selected only certain records to be processed. Of course, I can print these attributes out in my notebook too. I'll add another cell here. So I'm going to create a little variable called result, and I'll set result = sturgisHomes.query. Again, I'm going to put a where clause, and the where clause is going to equal that 1=1 again, so I get all the records. I'll just ask for the outfields to be just the OBJECTID. I'm just going to return two columns, not the whole table. I'll return the OBJECTID and the Inspector. Okay, so again, the syntax, right? It's going to take a while to get used to this. You just got to keep referring to the help. So this query takes as its first parameter a where clause, saying, which records do you want to pull back? That's specifically the query on a feature layer, okay? The query on a feature layer, you need this where clause as the first parameter, and then the outfields parameter lets you choose which columns you want to bring back. And the outfields doesn't accept a dictionary, it just accepts a comma-separated list of values. And again, you wouldn't know that, except to go check the help. So I've got that result. So then I'll print them out. For eachFeature in result.features, I'm going to print, and I'm going to use a little formatted print statement here. So I'll print the literal words, ObjectID:, and then I'll put curly braces. This is not a dictionary curly brace. This is a curly brace in this formatted print statement to say, "Hey, don't protect this value and don't escape it out and get its variable value." So I'm going to do eachFeature.attributes, right? So I'm getting the each feature object, I'm getting it's .attributes property, and I'm pulling out the OBJECTID key, close curly braces, the OBJECTID key from the attributes dictionary. So when I call that key, I'll get its value, right? I'll also put the word, Inspector, and a :, and I'll go out and grab that eachFeature.attributes["Inspector"], the inspector key. Close curly braces, close quotes, close parentheses. It's a lot of syntax, so you got to take your time, make sure you get it right. Now, I'll run this cell. Okay, let's see where I missed a square bracket somewhere. Oops, I made a little mistake there because I used double quotes around this Inspector, but I've already got double quotes being used to quote the entire string, so I need to put single quotes in there. So let's try it one more time. Okay, so there you go. There it's printed out the records, and you can see the inspector is Alphonso for all the different ObjectIDs. That's a little slow and clunky, looping through them all. I could use another Python module called Pandas to print this out a little nicer. It's a little bit easier to work with. I'll show you how to use the Pandas DataFrame here, but you'll have to save the details for another video. I'll query the results, just like before, but then I'll convert those results to a DataFrame. And then I can request any fields from that DataFrame. Watch. I'll say, import pandas, and I'll just call it as pd, just for a shorthand, a little alias. I'll query the features. Result = sturgisHomes query. Again, put that weird little where clause, where = "1=1". And then for outfields, this time I'm just going to choose all of them. So I put a * in there, choose all the outfields. And then I convert those results, that result variable, to a DataFrame. And I store that in a DataFrame variable. So I'll say df = result.sdf, because these feature sets that get returned from a query have a spatial DataFrame property that returns the data as a spatial DataFrame. That's just like a little table that you can easily visualize. So if I want to see like just the inspector column or any others, I'll say print the DataFrame, the DataFrame variable, the Inspector column, and the condition column. Now, I need two little sets of square brackets in there. And again, you can find out more about Pandas in another class, but I do want to mention that it's pretty easy to use. I need those two sets of brackets here because DataFrame needs a set of brackets around a column, and then I'm passing in a list of columns. So see it prints it out kind of nice, a little bit easier. And it's printing the ObjectID for me, by the way., By the way, I can also use that DataFrame to print the number of records by just printing the length of that DataFrame object. Print the length of the DataFrame. You can see there are 660 records. And you notice that print of the DataFrame only printed the first five and the last five records by default. So again, if you're interested in Pandas, look into the Pandas training, and you can find out more about working with these actual DataFrames, getting all the records printed out. So you see, once I have the feature layer object, I can work with it in a lot of ways. I can display it on a map, I can use it for geo processing, I can change the table schema, and I can even make edits to the data. There you go.