cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
299 Views
Registered: ‎01-25-2012

Synthesis crashes when using large bit vectors

Initially this was going to be posted as a question but while preparing a test case to post here, I stumbled across the solution. This post is to help others struggling with synthesis crashing out with useless messages such as:

/tools/Xilinx/Vivado/2019.2/bin/loader: line 280: 6759 Killed "$RDI_PROG" "$@"

Setup: Vivado 2019.2, Ubuntu 18.04.3 LTS (yes I know it is not officially supported yet but clearly this was not the problem), system RAM 64G, target device Alveo 250 board.

I tried all the solutions posted in the forum but with no success. I did manage to track the problem to particular module using the suggestions in https://forums.xilinx.com/t5/Design-and-Debug-Techniques-Blog/Vivado-Synthesis-Crash-Debugging-Guide/ba-p/946862.

The problematic module is converting a block of 5760 dwords into 5760 serial outputs (transmitted MSB to LSB). To make it run fast it was necessary to load all the data from brams into an array of 5760x32 FF's. Depending on the how you implement this in RTL you will get very different results. In the examples shown below, I ran OOC synthesis using the default settings. 

 

library ieee;
use IEEE.std_logic_1164.ALL;
use IEEE.numeric_std.ALL;

entity transpose_ex2 is
  Port (
    clk                   : in  std_logic;
    data_i                : in std_logic_vector(512-1 downto 0);
    load_en_i             : in std_logic_vector(360-1 downto 0);
    data_o               : out std_logic_vector(360*512-1 downto 0)
  );
end transpose_ex2;

architecture Behavioral of transpose_ex2 is

begin

  process (clk)
  begin
    if (rising_edge(clk)) then
      for i in 0 to 360-1 loop
        for j in 0 to 512-1 loop
          if load_en_i(i) = '1' then
            data_o(i*512+j) <= data_i(j);
          end if;
        end loop;
      end loop;
    end if;
  end process;

end Behavioral;

 

 Note that I wasn't even trying to get the serial output in this example (doing so doesn't make it any better). This example crashed synthesis after 3:30 hours. The log:

 

*** Running vivado
    with args -log transpose_ex2.vds -m64 -stack 2000 -product Vivado -mode batch -messageDb vivado.pb -notrace -source transpose_ex2.tcl


****** Vivado v2019.2 (64-bit)
  **** SW Build 2708876 on Wed Nov  6 21:39:14 MST 2019
  **** IP Build 2700528 on Thu Nov  7 00:09:20 MST 2019
    ** Copyright 1986-2019 Xilinx, Inc. All Rights Reserved.

Sourcing tcl script '/home/pspear/.Xilinx/Vivado/Vivado_init.tcl'
source transpose_ex2.tcl -notrace
Command: synth_design -top transpose_ex2 -part xcu250-figd2104-2L-e -mode out_of_context
Starting synth_design
Attempting to get a license for feature 'Synthesis' and/or device 'xcu250'
INFO: [Common 17-349] Got license for feature 'Synthesis' and/or device 'xcu250'
INFO: [Device 21-403] Loading part xcu250-figd2104-2L-e
INFO: Launching helper process for spawning children vivado processes
INFO: Helper process launched with PID 6903 
---------------------------------------------------------------------------------
Starting RTL Elaboration : Time (s): cpu = 00:00:03 ; elapsed = 00:00:04 . Memory (MB): peak = 4688.305 ; gain = 183.719 ; free physical = 49692 ; free virtual = 62304
---------------------------------------------------------------------------------
INFO: [Synth 8-638] synthesizing module 'transpose_ex2' [/home/pspear/work/testCase/testCase.srcs/sources_1/imports/vhdl/transpose_P16_ex2.vhd:17]
/tools/Xilinx/Vivado/2019.2/bin/loader: line 280:  6759 Killed                  "$RDI_PROG" "$@"

 

Interestingly, just changing from inference to instantiation and the code works.

 

library ieee;
use IEEE.std_logic_1164.ALL;
use IEEE.numeric_std.ALL;
Library UNISIM;
use UNISIM.vcomponents.all;

entity transpose_ex1 is
  Port (
    clk                   : in  std_logic;
    data_i                : in std_logic_vector(512-1 downto 0);
    load_en_i             : in std_logic_vector(360-1 downto 0);
    data_o               : out std_logic_vector(360*512-1 downto 0)
  );
end transpose_ex1;

architecture Behavioral of transpose_ex1 is

begin
  gen_depth: for i in 0 to 360-1 generate
    gen_width: for j in 0 to 512-1 generate
      FDRE_inst : FDRE
      port map (
        Q => data_o(i*512+j),
        C => clk,
        CE => load_en_i(i),
        D => data_i(j),
        R => '0'
      );
    end generate;
  end generate;

end Behavioral;

 

Synthesis took only 12 minutes. I won't show the log file since the tools do insist on printing out the parameters for each and every one of the 184k flipflops. I wasted a few hours looking into suppressing those messages in case that was what was causing the problem. It turns out you can't and that wasn't the problem. sigh....

If I try to extract the serial data slice, the tool once again failed. The following was added to the previous example and the data_o size was adjusted accordingly.

 

  process (clk)
  begin
    if (rising_edge(clk)) then
      for i in 0 to 5760-1 loop
        data_o(i) <= flat_reg(i*32+to_integer(unsigned(addr_i(4 downto 0))));
      end loop;
    end if;
  end process;

 

Synthesis crashed in little over an hour. The log file contained 737280 lines of useless parameter information abruptly terminating with the even more useless: 

/tools/Xilinx/Vivado/2019.2/bin/loader: line 280: 6754 Killed "$RDI_PROG" "$@"
Parent process (pid 6754) has died. This helper process will now exit

To make a long story a bit shorter, the problem is caused by big flat vectors. The following vector will cause the tools to fail

 

signal flat_reg: std_logic_vector(512*360-1 downto 0):= (others => '0');

 

but this won't. 

 

  type big_array_type is array(0 to 16*360-1) of std_logic_vector(31 downto 0);
  signal big_array : big_array_type := (others => (others => '0'));

 

The moral of the story is that just because it is easier to do the index math one way doesn't make it the best way. Here is the module that works. 

 

library ieee;
use IEEE.std_logic_1164.ALL;
use IEEE.numeric_std.ALL;

entity transpose_ex6 is
  Port (
    clk                   : in  std_logic;
    data_i                : in std_logic_vector(512-1 downto 0);
    load_en_i             : in std_logic_vector(360-1 downto 0);
    addr_i                : in std_logic_vector(4 downto 0);
    data_o                : out std_logic_vector(5760-1 downto 0)
  );
end transpose_ex6;

architecture Behavioral of transpose_ex6 is

  type big_array_type is array(0 to 16*360-1) of std_logic_vector(31 downto 0);
  signal big_array : big_array_type := (others => (others => '0'));

begin

  process (clk)
  begin
    if (rising_edge(clk)) then
      for i in 0 to 360-1 loop
        for j in 0 to 16-1 loop
          if load_en_i(i) = '1' then
            big_array(i*16+j) <= data_i((j+1)*32-1 downto j*32);
          end if;
        end loop;
      end loop;
    end if;
  end process;

  process (clk)
  begin
    if (rising_edge(clk)) then
      for i in 0 to 5760-1 loop
        data_o(i) <= big_array(i)(to_integer(unsigned(addr_i)));
      end loop;
    end if;
  end process;
end Behavioral;

 

 

Xilinx, could you please improve the error reporting and crash handling in synthesis. I've wasted a week messing around with this - ultrafast methodology ... ha

I've posted the archived test cases project so that you can reproduce the problem (in order to make the archive small enough to post here, I did have to strip out those useless parameter messages in all the runme.log and *.vds files in the OOC run folders. I also deleted the large vivado.pb and design check point files in the same locations).

0 Kudos
2 Replies
Highlighted
Moderator
Moderator
224 Views
Registered: ‎07-21-2014

Re: Synthesis crashes when using large bit vectors

pspear@dwavesys.com 

Thanks for reporting this bug and glad to know the blog was helpful for reducing the testcase.

I am able to reproduce the issue and will be filing a CR on this issue to get it fixed in future releases. Based on my understanding from the above post, I see that you already have a temporary workaround for this issue, please let me know if this is not correct.

After filing the CR and analyzing the issue further, I will be updating the thread again.

Thanks
Anusheel 

Highlighted
218 Views
Registered: ‎01-25-2012

Re: Synthesis crashes when using large bit vectors

Thanks for the fast response. The workaround is letting me make progress once again. Now if only I could solve my congestion problems...

0 Kudos