Streaming live at 10am (PST)

CSV Import With Multi-Reference Fields

Hello All,

I have a .csv that I am trying to import that has two columns for multi-reference fields (category and department).

These multi-reference fields are used to filter collection items based on the department and then by category to organize the dynamic lists into sections on a page.

I can’t see a way to “map” which collection item relates to which department or category on import as the system has the multi-reference fields grayed out and won’t let me map my .csv fields to them.

Am I missing something?

1 Like

Hi @EOC, Mapping for those fields is not available at this time. However, that would be an awesome improvement to have for that feature, and is something you could definitely post about in our Webflow Wishlist.

I hope this helps

Hello @cyberdave thanks for the quick reply, that’s a bummer to say the least. :cry:

We were thinking about migrating some WordPress clients in the near future, but how is this a truly viable option for migrating blog content if you can’t pull multi-reference fields? All the categories would be left behind for manual input later… Doesn’t the defeat the purpose?

Also, you may want to make a note to someone to update the how to article on .csv import to mention this… I wasted a lot of time creating a .csv that won’t work.

3 Likes

Hello @cyberdave just read my last reply and hope it didn’t come across as snarky, wasn’t meant to :slight_smile:

Hi @EOC, no worries, you raise a valid point about the multi-reference fields. More support for this will likely come, but I do not have the timeline yet for that.

Multi-reference fields were not included to the first phase of CSV import and now that this feature is in production, it will continue to be updated and improved.

Many of the features now in Webflow have been requested in the past by customers who have trusted Webflow to build the features they need. Stay tuned!

1 Like

Hello @cyberdave for the record, I think you guys are doing a great job and I look forward to new features and updates!

2 Likes

Yep. Hitting this wall as well. To the Wishlist!

1 Like

Wonder if there are any updates on this. I am trying to import 150 films with all data for a festival, and would need to map the films to Tag and Series collections. So for instance one film would come with three tags, that would already be in another collection in webflow.

Can I multi refer these automatically? Or do we need a couple of days extra budget to map them manually :face_with_raised_eyebrow:

cc/ @cyberdave

We solved the issue by importing without the references and afterwards using the Webflow API to update the reference fields. Was a bit of manual work, but worked just fine.

1 Like

@andre9000 thanks for the news of a workaround on this. It’s been a major issue for us too when creating CMS content from external files. Can I ask how you did it?? You did one pass that imported the .csv with the reference fields left blank - then what? Did you use zapper or any other automation in your process? Any guidance on this would be much appreciated. Cheers Dylan

I only used the webflow api with node:
https://developers.webflow.com/

You’ll need to fetch the item id of the item you want to reference.

we migrated several webflow collections into 1. the steps were the following:

  1. export collections into 1 standardized csv using the api. the reference id was an extra field “reference id”
  2. import the csv to the new collection
  3. setting references using the api and the id value stored in “reference id”

Hope that helps :slight_smile:

2 Likes

Hi @andre9000
I’m trying to do something like this, I have about more 1,000 items in a collection, each has 4 multi-reference field, I’ve already loaded my csv without the multireference.

I don’t understand how to download the csv from api, because I can only get a json file from that, non un csv.

I already can see the id’s of my multirefence from the API.

I need to understand how to download rewrite the file and reload it.

You could better explain the process you have used? could be the only solution to my problem.

thanks for your help in advance
Matt

@AMG_Blomat
Hi Matt,

do I understand correctly: You already imported all 1000 items to a webflow collection, and now just need to set the multireferences?

Or do you want to export it, for a non-webflow CMS?

first one, already imported all 1000 items to a webflow collection, and now just need to set the multireferences

I would probably not reupload everything via CSV, but just write a script that updates all existing items via webflow API.

Steps would go roughly like this:

  • authorize api
  • fetch an array of ids, one for every item in the collection
  • iterate through the array
    • define the fields and values that should change (the references)
    • update item via patch method

keep the api’s rate limit in mind.

3 Likes

Sorry but I have a lot of problem with update the data,
Now my workflow in node is:

1 - Download all the items with the API in one big json file.

2 - convert the json in csv and edit in excel

3 - convert to JSON and update via API (but I have a big problem with the rate limit)

The problem is the updateItem() method work single item, one by one,
it’s possible to give all the items of the JSON in one single API call?

Probably the patch method you mentioned but how it’s work?

one possibility would be to reattempt the request every few seconds, until it succeeds. We recently did a bulk update for several collections with this code, it dealt with the rate limit just fine:

// map: obj with changes
async function updateAllItems({ webflow, collectionId, map }) {
  const ids = await getAllItemIds({ webflow, collectionId });
  for (const [i, itemId] of ids.entries()) {
    try {
      await updateItem({
        webflow,
        collectionId,
        itemId,
        map
      });
    } catch (err) {
      console.error(`Error at index ${i}:`);
      console.error(err);
    }
  }
}

async function updateItem({
  webflow, collectionId, itemId, map
}) {
  try {
    // currently not implemented in webflow-api package
    const url = `https://api.webflow.com/collections/${collectionId}/items/${itemId}`;
    const options = {
      body: JSON.stringify({ fields: map }),
      method: 'PATCH',
      headers: {
        Authorization: `Bearer ${webflow.token}`,
        'Accept-Version': '1.0.0',
        'Content-Type': 'application/json'
      }
    };
    const response = await fetch(url, options);
    checkStatus(response);
    console.log(`✔ Updated ${itemId}`);
  } catch (err) {
    if (err.message === 'Too Many Requests') {
      console.log('too many requests');
      const waitingTime = 1000 * 5; // 5s
      wait(waitingTime);
      console.log('retry...');
      updateItem(...arguments);
    } else {
      throw err;
    }
  }
}
1 Like

(obiously there are some other functions being used, but they are either not important, or it is obvious what they are supposed to do)

thanks, seems clear what that function has to do, but I’m a real noob with javascript and json,
can I see how you declare this variables?

webflow,
collectionId,
itemId,
map <-- this is the json of all field I want to change correct?

I’m trying to define that variable, but I still get always this error :frowning:

(node:4297) UnhandledPromiseRejectionWarning: TypeError: Cannot destructure propertywebflowof 'undefined' or 'null'. at updateAllItems (/Users/mattia/Downloads/cms/update-all.js:14:30) at Object.<anonymous> (/Users/mattia/Downloads/cms/update-all.js:62:1) at Module._compile (internal/modules/cjs/loader.js:701:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10) at Module.load (internal/modules/cjs/loader.js:600:32) at tryModuleLoad (internal/modules/cjs/loader.js:539:12) at Function.Module._load (internal/modules/cjs/loader.js:531:3) at Function.Module.runMain (internal/modules/cjs/loader.js:754:12) at startup (internal/bootstrap/node.js:283:19) at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3) (node:4297) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:4297) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

The file I copied this from is much bigger, so this is all just parts of it, probably needs some changes to be able to run :grimacing:

map <-- this is the json of all field I want to change correct?

Yes!

For ‘webflow’, i use webflows npm package

you can call the functions from above like this:

import Webflow from 'webflow-api';
async function run() {
  try {
    const webflow = new Webflow({ token: YOUR_WEBFLOW_API_TOKEN });
    const collectionId = YOUR_COLLECTION_ID;
    const map = {
      'heading': 'My new heading'
    };
    await updateAllItems({
      webflow,
      collectionId,
      map
     });
   } catch (err) {
      console.error(err);
   }
}

run();
1 Like